Pretraining a language machine-learning model
US12236206B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | May 26, 2021 |
| Grant date | Feb 25, 2025 |
| Priority date | — |
| Expiry date | Aug 28, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/084
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
In one embodiment, a method includes accessing a first document, accessing a plurality of second documents, calculating a relevance score for each of the plurality of second documents indicating a degree of relevance of the second document to the first document using an encoder of a machine-learning model, selecting a subset of the second documents based on their corresponding relevance scores, generating a target document by using the machine-learning model to process the subset of second documents and their corresponding relevance scores, and updating parameters of the machine-learning model based on a comparison between the first document and the generated target document.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.