Method for pre-training language model
US12373735B2 · kind B2 · utility
0Cited by
0References
10Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Mar 7, 2023 |
| Grant date | Jul 29, 2025 |
| Priority date | — |
| Expiry date | Dec 26, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/289
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method for pre-training a language model includes: constructing a pre-training language data set, in which the pre-training language data set comprises unsupervised language data and supervised language data; generating a hierarchical multi-template and multi-task language data set based on the pre-training language data set; and pre-training the language model based on the hierarchical multi-template and multi-task language data set.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.