Hierarchical partitioning of operators

US12182688B2 · kind B2 · utility

Assignee

• AMAZON TECHNOLOGIES, INC. · US

Inventors

• Animesh Jain · Sunnyvale, US
• Yizhi Liu · Fremont, US
• Hongbin Zheng · San Jose, US
• Jeffrey T. Huynh · San Jose, US
• Haichen Li · San Jose, US
• Drazen Borkovic · La Cañada Flintridge, US
• Jindrich Zejda · Saratoga, US
• Richard John Heaton · San Jose, US
• Randy Renfu Huang · معلمی نژاد, US
• Zhi Chen · Shanghai, CN
• Yida Wang · Palo Alto, US

Key dates

• Filing date: Nov 27, 2019
• Grant date: Dec 31, 2024
• Expiry date: Feb 7, 2043

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/045
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Methods and apparatuses for hierarchical partitioning of operators of a neural network for execution on an acceleration engine are provided. Neural networks are built in machine learning frameworks using neural network operators. The neural network operators are compiled into executable code for the acceleration engine. Development of new framework-level operators can exceed the capability to map the newly developed framework-level operators onto the acceleration engine. To enable neural networks to be executed on an acceleration engine, hierarchical partitioning can be used to partition the operators of the neural network. The hierarchical partitioning can identify operators that are supported by a compiler for execution on the acceleration engine, operators to be compiled for execution on a host processor, and operators to be executed on the machine learning framework.