Compression of models for natural language processing
US12417357B1 · kind B1 · utility
0Cited by
0References
18Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Aug 8, 2022 |
| Grant date | Sep 16, 2025 |
| Priority date | — |
| Expiry date | Apr 14, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N20/00
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An example electronic computing device can include: a processor; and a system memory, the system memory including instructions which, when executed by the processor, cause the electronic computing device to: receive a model for natural language processing of data, the model including a plurality of self-attention heads; prune the model by removing one or more of the plurality of self-attention heads of the model to create a pruned model; and evaluate a classification accuracy of the pruned model to maintain a performance level.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.