Orthogonally constrained multi-head attention for speech tasks
US11908457B2 · kind B2 · utility
0Cited by
1References
28Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Jul 3, 2020 |
| Grant date | Feb 20, 2024 |
| Priority date | — |
| Expiry date | Jan 12, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2015/088
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method for operating a neural network includes receiving an input sequence at an encoder. The input sequence is encoded to produce a set of hidden representations. Attention-heads of the neural network calculate attention weights based on the hidden representations. A context vector is calculated for each attention-head based on the attention weights and the hidden representations. Each of the context vectors correspond to a portion of the input sequence. An inference is output based on the context vectors.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.