Training sequence generation neural networks using quality scores
US11699074B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 17, 2020 |
| Grant date | Jul 11, 2023 |
| Priority date | — |
| Expiry date | Dec 15, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N5/01
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a sequence generation neural network. One of the methods includes obtaining a batch of training examples; for each of the training examples: processing the training network input in the training example using the neural network to generate an output sequence; for each particular output position in the output sequence: identifying a prefix that includes the system outputs at positions before the particular output position in the output sequence, for each possible system output in the vocabulary, determining a highest quality score that can be assigned to any candidate output sequence that includes the prefix followed by the possible system output, and determining an update to the current values of the network parameters that increases a likelihood that the neural network generates a system output at the position that has a high quality score.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.