Recurrent neural network architectures which provide text describing images
US10949744B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 10, 2019 |
| Grant date | Mar 16, 2021 |
| Priority date | — |
| Expiry date | Jul 10, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/205
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Provided are systems and techniques that provide an output phrase describing an image. An example method includes creating, with a convolutional neural network, feature maps describing image features in locations in the image. The method also includes providing a skeletal phrase for the image by processing the feature maps with a first long short-term memory (LSTM) neural network trained based on a first set of ground truth phrases which exclude attribute words. Then, attribute words are provided by processing the skeletal phrase and the feature maps with a second LSTM neural network trained based on a second set of ground truth phrases including words for attributes. Then, the method combines the skeletal phrase and the attribute words to form the output phrase.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.