Method of bidirectional image-text retrieval based on multi-view joint embedding space
US11106951B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 29, 2018 |
| Grant date | Aug 31, 2021 |
| Priority date | — |
| Expiry date | May 26, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V30/274
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A bidirectional image-text retrieval method based on a multi-view joint embedding space includes: performing retrieval with reference to a semantic association relationship at a global level and a local level, obtaining the semantic association relationship at the global level and the local level in a frame-sentence view and a region-phrase view, and obtaining semantic association information in a global level subspace of frame and sentence in the frame-sentence view, obtaining semantic association information in a local level subspace of region and phrase in the region-phrase view, processing data by a dual-branch neural network in the two views to obtain an isomorphic feature and embedding the same in a common space, and using a constraint condition to reserve an original semantic relationship of the data during training, and merging the two semantic association relationships using multi-view merging and sorting to obtain a more accurate semantic similarity between data.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.