Sequence processing using online attention
US11080589B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 8, 2019 |
| Grant date | Aug 3, 2021 |
| Priority date | — |
| Expiry date | Jul 8, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/09
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence including a respective output at each of multiple output time steps from respective encoded representations of inputs in an input sequence. The method includes, for each output time step, starting from the position, in the input order, of the encoded representation that was selected as a preceding context vector at a preceding output time step, traversing the encoded representations until an encoded representation is selected as a current context vector at the output time step. A decoder neural network processes the current context vector and a preceding output at the preceding output time step to generate a respective output score for each possible output and to update the hidden state of the decoder recurrent neural network. An output is selected for the output time step using the output scores.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.