Methods, apparatuses, and computer program products using a repeated convolution-based attention module for improved neural network implementations
US11651191B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Sep 3, 2019 |
| Grant date | May 16, 2023 |
| Priority date | — |
| Expiry date | Jan 22, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/084
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method, apparatus, and computer program product are provided for providing improved neural network implementations using a repeated convolution-based attention module. Example embodiments implement a repeated convolution-based attention module that utilizes multiple iterations of a repeated convolutional application layer and subsequent augmentations to generate an attention module output. Example methods may include augmenting an attention input data object based on a previous iteration convolutional output to produce a current iteration input parameter, inputting the input parameter to a repeated convolutional application layer to generate a current iteration input parameter, repeating for multiple iterations, and augmenting the attention input data object based on the final convolutional output to produce an attention module output. Other methods may include an initial convolutional application layer, and/or apply and augment the output of the initial convolutional application layer, and include convolutional application layer(s) having at least two sub-layers.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.