Language identification in multilingual text
US8635061B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 14, 2010 |
| Grant date | Jan 21, 2014 |
| Priority date | — |
| Expiry date | Mar 31, 2031 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/951
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and media are provided for identifying languages in multilingual text. A document is decoded into a universal representative coding for easier tag manipulation, then broken into plain-text content sections. The sections are identified and assigned a weight, wherein more informative sections are given a higher weight and less informative sections are given a lesser weight. A language likelihood score is determined for each word, phrase, or character n-gram in a section. The language likelihood scores within a section are combined for each language. The combined section scores are then summed together to obtain a total document score for each language. This results in a document score for each language, which can be ranked to determine the primary language for the document.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.