Ordering computations of a machine learning network in a machine learning accelerator for efficient memory usage
US11803740B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Feb 8, 2023 |
| Grant date | Oct 31, 2023 |
| Priority date | — |
| Expiry date | Feb 8, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/045
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A compiler manages memory usage in the machine learning accelerator by intelligently ordering computations of a machine learning network. The compiler identifies partial networks of the machine learning network representing portions of the machine learning network across multiple layers on which an output or set of outputs are dependent. Because any given output may depend on only a limited subset of intermediate outputs from the prior layers, each partial network may include only a small fraction of the intermediate outputs from each layer. Instead of implementing the MLN by computing one layer at a time, the compiler schedules instructions to sequentially implement partial networks. As each layer of a partial network is completed, the intermediate outputs can be released from memory. The described technique enables intermediate outputs to be directly streamed between processing elements of the machine learning accelerator without requiring large transfers to and from external memory.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.