Speculative decoding in autoregressive generative artificial intelligence models
US12229192B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 13, 2023 |
| Grant date | Feb 18, 2025 |
| Priority date | — |
| Expiry date | Dec 13, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/284
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Certain aspects of the present disclosure provide techniques and apparatus for generating a response to a query input in a generative artificial intelligence model. An example method generally includes receiving a plurality of sets of tokens generated based on an input prompt and a first generative artificial intelligence model, each set of tokens in the plurality of sets of tokens corresponding to a candidate response to the input prompt; selecting, using a second generative artificial intelligence model and recursive adjustment of a target distribution associated with the received plurality of sets of tokens, a set of tokens from the plurality of sets of tokens; and outputting the selected set of tokens as a response to the input prompt.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.