Processing text sequences using neural networks
US10733390B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 7, 2019 |
| Grant date | Aug 4, 2020 |
| Priority date | — |
| Expiry date | Jun 7, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L15/197
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for language modeling. In one aspect, a system comprises: a masked convolutional decoder neural network that comprises a plurality of masked convolutional neural network layers and is configured to generate a respective probability distribution over a set of possible target embeddings at each of a plurality of time steps; and a modeling engine that is configured to use the respective probability distribution generated by the decoder neural network at each of the plurality of time steps to estimate a probability that a string represented by the target embeddings corresponding to the plurality of time steps belongs to the natural language.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.