Graph rewriting for large model support using categorized topological sort
US10884755B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 31, 2019 |
| Grant date | Jan 5, 2021 |
| Priority date | — |
| Expiry date | Jul 31, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T11/206
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A computer-implemented method is provided for managing GPU memory consumption by computational graph rewriting. The method includes constructing, by a hardware processor, a categorized topological ordering of a computational graph. The categorized topological ordering includes multiple computational nodes arranged in multiple levels. The method further includes estimating, by the hardware processor, the GPU memory consumption responsive to a level including two or more computational nodes from among the multiple computational nodes. The method also includes rewriting, by the hardware processor, the computational graph by linearizing the two or more computational nodes in the level to avoid overlapping of the GPU memory consumption by the two or more computational nodes responsive to the GPU memory consumption exceeding a threshold. The memory additionally includes managing the GPU memory consumption in accordance with the rewritten computational graph.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.