Method and apparatus for acquiring pre-trained model
US12277401B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 15, 2021 |
| Grant date | Apr 15, 2025 |
| Priority date | — |
| Expiry date | Sep 20, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/02
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
The present disclosure discloses a method and apparatus for acquiring a pre-trained model, and relates to natural language processing and deep learning technologies in the field of artificial intelligence technologies. An implementation includes: acquiring training data, the training data including a single-modal language material and a multi-modal language material, and the multi-modal language material including a language material pair formed by a first-modal language material and a second-modal language material; and performing a multi-task training operation on a pre-trained model using the training data, the multi-task including at least one cross-modal contrastive learning task and at least one single-modal learning task; the pre-trained language model obtained in the present disclosure may learn from different forms of language materials, i.e., the single-modal language material and the multi-modal language material, such that the pre-trained language model may effectively process information in various modals.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.