Language autodetection from non-character sub-token signals
US11630951B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 13, 2022 |
| Grant date | Apr 18, 2023 |
| Priority date | — |
| Expiry date | Jun 13, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/263
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
In non-limiting examples of the present disclosure, systems, methods and devices for determining a language of a text string are presented. A language detection model may be maintained. The language detection model may comprise identities and weights for initial and final consonants, identities and weights for prefixes and suffixes, and identities and weights for vowel sequences, where each identity is derived from a training corpus. The weights may correspond to a frequency of a text unit in the corpus. A text string may be received and a match score between the text string and the language of the language detection model may be determined. The match score may be based on initial and final consonant scores, prefix and suffix scores, and/or vowel sequence scores for each word in the text string. If the match score meets a threshold value a follow-up action associated with the language may be performed.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.