Neural network layer processing with normalization and transformation of data
US11562201B2 · kind B2 · utility
Assignee
Inventor
Key dates
| Filing date | Feb 25, 2019 |
| Grant date | Jan 24, 2023 |
| Priority date | — |
| Expiry date | Nov 26, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/08
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Processors and methods for neural network processing are provided. A method includes receiving a subset of data corresponding to a layer of a neural network for processing using the processor. The method further includes during a forward propagation pass: (1) normalizing the subset of the data corresponding to the layer of the neural network based on an average associated with the subset of the data and a variance associated with the subset of the data, where the normalizing the subset of the data comprises dynamically updating the average and dynamically updating the variance, to generate normalized data and (2) applying a transformation to the normalized data using a fixed scale parameter corresponding to the subset of the data and a fixed shift parameter corresponding to the subset of the data such that during the forward propagation pass neither the fixed scale parameter nor the fixed shift parameter is updated.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.