High precision low bit convolutional neural network
US10824943B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 21, 2018 |
| Grant date | Nov 3, 2020 |
| Priority date | — |
| Expiry date | Jan 5, 2039 |
Classification
- Technology area (CPC Y)Emerging Cross-Sectional Technologies
- CPC primaryY02T10/40
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.