Memory efficiency for convolutional neural networks operating on graphics processing units
US10489703B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 20, 2016 |
| Grant date | Nov 26, 2019 |
| Priority date | — |
| Expiry date | Jul 8, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/955
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Aspects of the present disclosure are directed to techniques that improve performance of CNN systems through the effect of improved memory efficiencies for CNNs operating on GPUs. Aspects of the disclosure demonstrate that off-chip memory in such CNN systems is underutilized due to at least three characteristics namely, data layout, data locality and inter-kernel redundancy. Aspects of the disclosure examine the performance impact of different data layouts and then describe a method to produce data layout selection for various layers of the CNN including a fast transformation implementation. Disclosed are improvements to data locality from working set expansion, elimination of inter-kernel redundancy and increase of TLP using kernel reconstruction techniques including kernel fusion and thread injection. Disclosed experimental results show that our optimizations are very effective to boost the performance of CNNs by amounts up to 9.76 times for a single kernel and 2.05 times for a network.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.