Machine translation using neural network models

US11809834B2 · kind B2 · utility

Assignee

• Google LLC · US

Inventors

• Zhifeng Chen · Sunnyvale, US
• Macduff Richard Hughes · Los Gatos, US
• Yonghui Wu · Fremont, US
• Michael Schuster · Saratoga, US
• Xu Chen · Ann Arbor, US
• Llion Owen Jones · San Francisco, US
• Niki J. Parmar · Sunnyvale, US
• George Foster · Ottawa, CA
• Orhan Firat · Mountain View, US
• Ankur Bapna · Sunnyvale, US
• Wolfgang Macherey · Sunnyvale, US
• Melvin Jose Johnson Premkumar · Sunnyvale, US

Key dates

• Filing date: Aug 27, 2021
• Grant date: Nov 7, 2023
• Expiry date: Dec 5, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/09
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for machine translation using neural networks. In some implementations, a text in one language is translated into a second language using a neural network model. The model can include an encoder neural network comprising a plurality of bidirectional recurrent neural network layers. The encoding vectors are processed using a multi-headed attention module configured to generate multiple attention context vectors for each encoding vector. A decoder neural network generates a sequence of decoder output vectors using the attention context vectors. The decoder output vectors can represent distributions over various language elements of the second language, allowing a translation of the text into the second language to be determined based on the sequence of decoder output vectors.