Optimized asynchronous training of neural networks using a distributed parameter server with eager updates
US11630994B2 · kind B2 · utility
0Cited by
2References
21Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Feb 17, 2018 |
| Grant date | Apr 18, 2023 |
| Priority date | — |
| Expiry date | Aug 4, 2040 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/098
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method of training a neural network includes, at a local computing node, receiving remote parameters from a set of one or more remote computing nodes, initiating execution of a forward pass in a local neural network in the local computing node to determine a final output based on the remote parameters, initiating execution of a backward pass in the local neural network to determine updated parameters for the local neural network, and prior to completion of the backward pass, transmitting a subset of the updated parameters to the set of remote computing nodes.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.