Parallel decoding using transformer models
US11681954B2 · kind B2 · utility
0Cited by
0References
18Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Nov 13, 2019 |
| Grant date | Jun 20, 2023 |
| Priority date | — |
| Expiry date | Apr 20, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/82
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing parallel generation of output from an autoregressive sequence to sequence model. In one aspect, a blockwise parallel decoding method takes advantage of the fact that some architectures can score sequences in sublinear time. By generating predictions for multiple time steps at once then backing off to a longest prefix validated by the scoring model, the methods can substantially improve the speed of greedy decoding without compromising performance.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.