Asynchronous stochastic gradient descent
US10628740B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 5, 2016 |
| Grant date | Apr 21, 2020 |
| Priority date | — |
| Expiry date | Feb 21, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/084
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
The example computer-implemented method may comprise computing, by a generator processor on each of a plurality of learners, a gradient for a mini-batch using a current weight at each of the plurality of learners. The method may also comprise generating, by the generator processor on each of the plurality of learners, a plurality of triples, wherein each of the triples comprises the gradient, the weight index of the current weights used to compute the gradient, and a mass of the gradient. The method may further comprise performing, by a reconciler processor on each of the plurality of learners, an allreduce operation on the plurality of triples to obtain an allreduced triple sequence. Additionally, the method may comprise updating, by the reconciler processor on each of the plurality of learners, the current weight at each of the plurality of learners to a new current weight using the allreduced triple sequence.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.