Knowledge distillation for neural networks using multiple augmentation strategies
US11610393B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 2, 2020 |
| Grant date | Mar 21, 2023 |
| Priority date | — |
| Expiry date | Jul 15, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/82
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and efficiently learning parameters of a distilled neural network from parameters of a source neural network utilizing multiple augmentation strategies. For example, the disclosed systems can generate lightly augmented digital images and heavily augmented digital images. The disclosed systems can further learn parameters for a source neural network from the lightly augmented digital images. Moreover, the disclosed systems can learn parameters for a distilled neural network from the parameters learned for the source neural network. For example, the disclosed systems can compare classifications of heavily augmented digital images generated by the source neural network and the distilled neural network to transfer learned parameters from the source neural network to the distilled neural network via a knowledge distillation loss function.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.