Training of large neural networks

US12353981B2 · kind B2 · utility

Assignee

• Google LLC · US

Inventors

• Slav Petrov · New York, US
• Yonghui Wu · Fremont, US
• Andrew M. Dai · San Francisco, US
• David Richard So · San Francisco, US
• Dmitry Lepikhin · Sunnyvale, US
• Erica Ann Moreira · Fremont, US
• Gaurav Mishra · Edmonton, CA
• Jonathan Hudson Clark · Seattle, US
• Maxim Krikun · Castro Valley, US
• Melvin Jose Johnson Premkumar · Sunnyvale, US
• Nan Du · Palo Alto, US
• Orhan Firat · Mountain View, US
• Rohan Anil · San Francisco, US
• Siamak Shakeri · New York, US
• Xavier Eduardo Garcia · New York, US
• Yanping Huang · Mountain View, US
• Yong Cheng · Singapore, SG
• Yuanzhong Xu · Calabasas, US
• Yujing Zhang · Sunnyvale, US
• Zachary Alexander Nado · Brookline, US
• Eric Jun Jie Ni · San Jose, US
• Kefan Xiao · Sunnyvale, US
• Vladimir Feinberg · San Francisco, US
• Jin Young Sohn · Jersey City, US
• Aurko Roy · San Francisco, US

Key dates

• Filing date: May 10, 2024
• Grant date: Jul 8, 2025
• Expiry date: May 10, 2044

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/088
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform any one or more of a variety of machine learning tasks. For example, the neural network can be configured as a generative neural network, e.g., an autoregressive generative neural network.