System and methods for performing NLP related tasks using contextualized word representations
US11030414B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 18, 2018 |
| Grant date | Jun 8, 2021 |
| Priority date | — |
| Expiry date | Aug 8, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/045
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems, apparatuses, and methods for representing words or phrases, and using the representation to perform NLP and NLU tasks, where these tasks include sentiment analysis, question answering, and conference resolution. Embodiments introduce a type of deep contextualized word representation that models both complex characteristics of word use, and how these uses vary across linguistic contexts. The word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. These representations can be added to existing task models and significantly improve the state of the art across challenging NLP problems, including question answering, textual entailment and sentiment analysis.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.