Compression of fully connected / recurrent layers of deep network(s) through enforcing spatial locality to weight matrices and effecting frequency compression
US11977974B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 30, 2017 |
| Grant date | May 7, 2024 |
| Priority date | — |
| Expiry date | May 7, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N20/10
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system, having a memory that stores computer executable components, and a processor that executes the computer executable components, reduces data size in connection with training a neural network by exploiting spatial locality to weight matrices and effecting frequency transformation and compression. A receiving component receives neural network data in the form of a compressed frequency-domain weight matrix. A segmentation component segments the initial weight matrix into original sub-components, wherein respective original sub-components have spatial weights. A sampling component applies a generalized weight distribution to the respective original sub-components to generate respective normalized sub-components. A transform component applies a transform to the respective normalized sub-components. A cropping component crops high-frequency weights of the respective transformed normalized sub-components to yield a set of low-frequency normalized sub-components to generate a compressed representation of the original sub-components.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.