Activation layers for deep learning networks
US9892344B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 30, 2015 |
| Grant date | Feb 13, 2018 |
| Priority date | — |
| Expiry date | Mar 9, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/454
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Tasks such as object classification from image data can take advantage of a deep learning process using convolutional neural networks. These networks can include a convolutional layer followed by an activation layer, or activation unit, among other potential layers. Improved accuracy can be obtained by using a generalized linear unit (GLU) as an activation unit in such a network, where a GLU is linear for both positive and negative inputs, and is defined by a positive slope, a negative slope, and a bias. These parameters can be learned for each channel or a block of channels, and stacking those types of activation units can further improve accuracy.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.