Heterogeneous ML accelerator cluster with flexible system resource balance

US12417047B2 · kind B2 · utility

Assignee

• Google LLC · US

Inventors

• Sheng Li · Hangzhou City, CN
• Sridhar Lakshmanamurthy · Sunnyvale, US
• Norman Paul Jouppi · Palo Alto, US
• Martin G. Dixon · Portland, US
• Daniel Stodolsky · Somerville, US
• Quoc V. Le · Stanford, US
• Liqun Cheng · San Jose, US
• Erik K. Norden · San Jose, US
• Parthasarathy Ranganathan · Palo Alto, US

Key dates

• Filing date: Jan 10, 2023
• Grant date: Sep 16, 2025
• Expiry date: Mar 15, 2043

Classification

• Technology area (CPC G): Physics
• CPC primary: G06F15/7896
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Aspects of the disclosure are directed to a heterogeneous machine learning accelerator system with compute and memory nodes connected by high speed chip-to-chip interconnects. While existing remote/disaggregated memory may require memory expansion via remote processing units, aspects of the disclosure add memory nodes into machine learning accelerator clusters via the chip-to-chip interconnects without needing assistance from remote processing units to achieve higher performance, simpler software stack, and/or lower cost. The memory nodes may support prefetch and intelligent compression to enable the use of low cost memory without performance degradation.