Multi-model training pipeline in distributed systems
US11468325B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Mar 30, 2020 |
| Grant date | Oct 11, 2022 |
| Priority date | — |
| Expiry date | Apr 9, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/098
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A first worker node of a distributed system computes a first set of gradients using a first neural network model and a first set of weights associated with the first neural network model. The first set of gradients are transmitted from the first worker node to a second worker node of the distributed system. The second worker node computes a first set of synchronized gradients based on the first set of gradients. While the first set of synchronized gradients are being computed, the first worker node computes a second set of gradients using a second neural network model and a second set of weights associated with the second neural network model. The second set of gradients are transmitted from the first worker node to the second worker node. The second worker node computes a second set of synchronized gradients based on the second set of gradients.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.