Loss-aware replication of neural network layers
US11847567B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 7, 2020 |
| Grant date | Dec 19, 2023 |
| Priority date | — |
| Expiry date | Jul 28, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N20/00
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Some embodiments provide a method that receives a network with trained floating-point weight values. The network includes layers of nodes, each of which computes an output value based on input values and trained weight values. To replace a first layer of the trained network in a modified network with quantized weight values, the method defines multiple replica layers. Each replica layer includes nodes that correspond to nodes of the first layer, has a different set of allowed quantized weight values, and receives the same input values from a previous layer of the modified network such that groups of corresponding nodes from the replica layers operate correspondingly to the first layer. The method trains the quantized weight values of the modified network using a loss function with terms that account for effect on the loss function due to the quantization and for interactions between corresponding weight values of the replica layers.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.