Systems and methods for unifying statistical models for different data modalities
US11494589B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Mar 8, 2021 |
| Grant date | Nov 8, 2022 |
| Priority date | — |
| Expiry date | Mar 8, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/044
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques for performing a prediction task using a multi-modal statistical model configured to receive input data from multiple modalities including input data from a first modality and input data from a second modality different from the first modality. The techniques include: obtaining information specifying the multi-modal statistical model including values of parameters of each of multiple components of the multi-modal statistical model, the multiple components including first and second encoders for processing input data for the first and second modalities, respectively, first and second modality embeddings, a joint-modality representation, and a predictor; obtaining first input data for the first data modality; providing the first input data to the first encoder to generate a first feature vector; identifying a second feature vector using the joint-modality representation, the first modality embedding and the first feature vector; and generating a prediction for the prediction task using the predictor and the second feature vector.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.