Method and apparatus for next token prediction based on previously observed tokens
US11043117B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 30, 2018 |
| Grant date | Jun 22, 2021 |
| Priority date | — |
| Expiry date | Jan 23, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG08G1/0133
- WIPO fieldControl
- WIPO sectorInstruments
Abstract
An approach is provided for next token prediction based on previously observed tokens. The approach involves receiving an observed time series of tokens, wherein each of the tokens represents an observed data pattern. The approach also involves adding a most recent token from the observed time series of tokens into a variable token set. The approach further involves processing a historical token set to determine a historical token sequence comprising the variable token set followed by a next token. The approach further involves recursively adding a next most recent token from the observed time series of tokens into the variable token set for processing until the next token following the variable token set in the determined historical token sequence is unique or meets a target number of possible predictions. The approach further involves presenting the next token as a predicted next token of the observed time series of tokens.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.