Multi-modal image ranking using neural networks
US10642887B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 27, 2016 |
| Grant date | May 5, 2020 |
| Priority date | — |
| Expiry date | Sep 15, 2037 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/09
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods and systems are provided for ranking images against queries. A visual modality ranking of visual features of a digital image against a query is generated. A language modality ranking of text features of text associated with the digital image against the query is also generated. A multi-modal neural network determines importance weightings of the language modality ranking and the visual modality ranking against the query. The visual modality ranking and the language modality ranking are combined into a multi-modal ranking of the digital image against the query based on the importance weightings. The digital image is provided as a search result of the query based on the multi-modal ranking.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.