Generating scene graphs from digital images using external knowledge and image reconstruction
US11373390B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 21, 2019 |
| Grant date | Jun 28, 2022 |
| Priority date | — |
| Expiry date | Oct 21, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/82
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and non-transitory computer readable storage media are disclosed for generating semantic scene graphs for digital images using an external knowledgebase for feature refinement. For example, the disclosed system can determine object proposals and subgraph proposals for a digital image to indicate candidate relationships between objects in the digital image. The disclosed system can then extract relationships from an external knowledgebase for refining features of the object proposals and the subgraph proposals. Additionally, the disclosed system can generate a semantic scene graph for the digital image based on the refined features of the object/subgraph proposals. Furthermore, the disclosed system can update/train a semantic scene graph generation network based on the generated semantic scene graph. The disclosed system can also reconstruct the image using object labels based on the refined features to further update/train the semantic scene graph generation network.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.