Streaming self-attention in a neural network
US11961514B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 10, 2021 |
| Grant date | Apr 16, 2024 |
| Priority date | — |
| Expiry date | Oct 19, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L25/30
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An acoustic event detection system may employ one or more recurrent neural networks (RNNs) to extract features from audio data, and use the extracted features to determine the presence of an acoustic event. The system may use self-attention to emphasize features extracted from portions of audio data that may include features more useful for detecting acoustic events. The system may perform self-attention in an iterative manner to reduce the amount of memory used to store hidden states of the RNN while processing successive portions of the audio data. The system may process the portions of the audio data using the RNN to generate a hidden state for each portion. The system may calculate an interim embedding for each hidden state. An interim embedding calculated for the last hidden state may be normalized to determine a final embedding representing features extracted from the input data by the RNN.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.