Efficient inter-chip interconnect topology for distributed parallel deep learning
US11922219B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 6, 2023 |
| Grant date | Mar 5, 2024 |
| Priority date | — |
| Expiry date | Jan 6, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/098
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Embodiments of the present disclosure present a hyper-square interconnect topology and advanced ring-based AllReduce operations. In some embodiments, a topology is provided that is an improvement over conventional interconnect topologies by eliminating delays associated with long wirings. In some embodiments, computing nodes are divided into sub-sections to better allocate computing tasks, and the system can be optimized to divide up the computing nodes by maximizing the number of square sub-sections in the topology. In some embodiments, the system can be optimized to select square sub-sections first for each computing task. Each sub-section can comprise some computing nodes or all computing nodes in the hyper-square interconnect topology. This flexibility allows the hyper-square interconnect topology to utilize the computing nodes more efficiently by assigning appropriate numbers of computing nodes to each computing task based on the computing need of the computing task.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.