Training sequence natural language processing engines
US10402752B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 18, 2016 |
| Grant date | Sep 3, 2019 |
| Priority date | — |
| Expiry date | Nov 18, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/047
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system for training a model to predict a sequence (e.g. a sequence of words) given a context is disclosed. A model can be trained to make these predictions using a combination of individual predictions compared to base truth and sequences of predictions based on previous predictions, where the resulting sequence is compared to the base truth sequence. In particular, the model can initially use the individual predictions to train the model. The model can then be further trained over the training data in multiple iterations, where each iteration includes two processes for each training element. In the first process, an initial part of the sequence is predicted, and the model and model parameters are updated after each prediction. In the second process, the entire remaining amount of the sequence is predicted and compared to the corresponding training sequence to adjust model parameters to encourage or discourage each prediction.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.