Method and apparatus with key-value coupling
US11295220B2 · kind B2 · utility
0Cited by
3References
19Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Aug 8, 2019 |
| Grant date | Apr 5, 2022 |
| Priority date | — |
| Expiry date | Nov 22, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/82
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A processor-implemented method of implementing an attention mechanism in a neural network includes obtaining key-value coupling data determined based on an operation between new key data determined using a first nonlinear transformation for key data of an attention layer, and value data of the attention layer corresponding to the key data; determining new query data by applying a second nonlinear transformation to query data corresponding to input data of the attention layer; and determining output data of the attention layer based on an operation between the new query data and the key-value coupling data.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.