Training a model using parameter server shards
US8768870B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 15, 2013 |
| Grant date | Jul 1, 2014 |
| Priority date | — |
| Expiry date | Aug 15, 2033 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N7/01
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a model using parameter server shards. One of the methods includes receiving, at a parameter server shard configured to maintain values of a disjoint partition of the parameters of the model, a succession of respective requests for parameter values from each of a plurality of replicas of the model; in response to each request, downloading a current value of each requested parameter to the replica from which the request was received; receiving a succession of uploads, each upload including respective delta values for each of the parameters in the partition maintained by the shard; and updating values of the parameters in the partition maintained by the parameter server shard repeatedly based on the uploads of delta values to generate current parameter values.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.