Tokenizing alphanumeric text through use of finite state machines
US12242807B2 · kind B2 · utility
0Cited by
15References
10Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Mar 2, 2021 |
| Grant date | Mar 4, 2025 |
| Priority date | — |
| Expiry date | Nov 5, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/30
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Described herein are technologies related to tokenizing alphanumeric text through use of a tokenization algorithm that is at least partially implemented as a finite state machine. The tokenization algorithm is configured to output numeric identifiers that represent tokens or sub-tokens in the alphanumeric text.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.