Communication optimizations for distributed machine learning
US11704565B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Mar 3, 2022 |
| Grant date | Jul 18, 2023 |
| Priority date | — |
| Expiry date | Mar 3, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N7/01
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Embodiments described herein provide a system to configure distributed training of a neural network, the system comprising memory to store a library to facilitate data transmission during distributed training of the neural network; a network interface to enable transmission and receipt of configuration data associated with a set of worker nodes, the worker nodes configured to perform distributed training of the neural network; and a processor to execute instructions provided by the library. The instructions cause the processor to create one or more groups of the worker nodes, the one or more groups of worker nodes to be created based on a communication pattern for messages to be transmitted between the worker nodes during distributed training of the neural network. The processor can transparently adjust communication paths between worker nodes based on the communication pattern.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.