Spoken query processing for image search
US12288549B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 15, 2022 |
| Grant date | Apr 29, 2025 |
| Priority date | — |
| Expiry date | Jul 15, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L15/16
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An image search system uses a multi-modal model to determine relevance of images to a spoken query. The multi-modal model includes a spoken language model that extracts features from spoken query and a language processing model that extract features from an image. The multi-model model determines a relevance score for the image and the spoken query based on the extracted features. The multi-modal model is trained using a curriculum approach that includes training the spoken language model using audio data. Subsequently, a training dataset comprising a plurality of spoken queries and one or more images associated with each spoken query is used to jointly train the spoken language model and an image processing model to provide a trained multi-modal model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.