System and method for semantic analysis of multimedia data using attention-based fusion network
US11423304B2 · kind B2 · utility
Assignees
Inventors
Key dates
| Filing date | Jan 15, 2020 |
| Grant date | Aug 23, 2022 |
| Priority date | — |
| Expiry date | Apr 9, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/82
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system and a method for multimedia data analysis. The system includes a computing device. The computer device includes a processor and a storage device storing computer executable code. The computer executable code, when executed at the processor, is configured to: encode task description and task meta info into concatenated task vectors; encode context text and context meta info into concatenated context text vectors; encode context image and the context meta info into concatenated context image vectors; perform dual coattention on the concatenated task vectors and the concatenated context text and image vectors to obtain attended task vectors and attended context vectors; perform BiLSTM on the attended task vectors and the attended context vectors to obtain task encoding and context encoding; and decode the task encoding and the context encoding to obtain an answer to the task.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.