Systems and methods for trace norm regularization and faster inference for embedded models
US11556775B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 3, 2018 |
| Grant date | Jan 17, 2023 |
| Priority date | — |
| Expiry date | Nov 14, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L15/16
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Described herein are systems and methods for compressing and speeding up dense matrix multiplications as found, for examples, in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR). For compression, trace norm regularization technique embodiments were introduced and studied for training low rank factored versions of matrix multiplications. Compared to standard low rank training, the methods more consistently lead to good accuracy versus number of parameter trade-offs and can be used to speed-up training of large models. Faster inference may be further enabled on ARM processors through kernels optimized for small batch sizes, resulting in speed ups over the currently used library. Beyond LVCSR, the techniques are also generally applicable to embedded neural networks with large fully connected or recurrent layers.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.