Automatically determining configurations for executing recurrent neural networks

US11769035B1 · kind B1 · utility

Assignee

• AMAZON TECHNOLOGIES, INC. · US

Inventors

• Lai Wei · Kowloon, CN
• Hagay Lupesko · San Mateo, US
• Anirudh R. Acharya · San Diego, US
• Ankit Khedia · Redwood City, US
• Sandeep Krishnamurthy · Kanchinakote, IN
• Cheng-Che Lee · Taichung, TW
• Kalyanee Chendke · Cupertino, US
• Vandana Kannan · Bengaluru, IN
• Roshani Nagmote · San Mateo, US

Key dates

• Filing date: Dec 13, 2018
• Grant date: Sep 26, 2023
• Expiry date: Jul 27, 2042

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/0985
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Techniques are described automatically determining runtime configurations used to execute recurrent neural networks (RNNs) for training or inference. One such configuration involves determining whether to execute an RNN in a looped, or “rolled,” execution pattern or in a non-looped, or “unrolled,” execution pattern. Execution of an RNN using a rolled execution pattern generally consumes less memory resources than execution using an unrolled execution pattern, whereas execution of an RNN using an unrolled execution pattern typically executes faster. The configuration choice thus involves a time-memory tradeoff that can significantly affect the performance of the RNN execution. This determination is made automatically by a machine learning (ML) runtime by analyzing various factors such as, for example, a type of RNN being executed, the network structure of the RNN, characteristics of the input data to the RNN, an amount of computing resources available, and so forth.