Multi-GPU deep learning using CPUs
US11164079B2 · kind B2 · utility
2Cited by
1References
18Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Dec 15, 2017 |
| Grant date | Nov 2, 2021 |
| Priority date | — |
| Expiry date | Sep 2, 2040 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T1/20
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A computer-implemented method, computer program product, and computer processing system are provided for accelerating neural network data parallel training in multiple graphics processing units (GPUs) using at least one central processing unit (CPU). The method includes forming a set of chunks. Each of the chunks includes a respective group of neural network layers other than a last layer. The method further includes performing one or more chunk-wise synchronization operations during a backward phase of the neural network data parallel training, by each of the multiple GPUs and the at least one CPU.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.