Generating multi modal image representation for an image
US11244205B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Mar 29, 2019 |
| Grant date | Feb 8, 2022 |
| Priority date | — |
| Expiry date | Mar 29, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V30/10
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Technologies for generating a multi-modal representation of an image based on the image content are provided. The disclosed techniques include receiving an image, to be classified, that comprises one or more embedded text characters. The one or more embedded text characters are identified from the image and a first machine learning model is used to generate a text vector that represents a numerical representation of the one or more embedded text characters. A second machine learning model is used to generate an image vector that represents a numerical representation of the graphical portion of the image. The text vector and the image vector are used as input to generate a multi-modal vector that contains information from both the text vector and the image vector. The image may be classified into one of a plurality of image classifications based upon the information in the multi-modal vector.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.