Systems and methods for neural language modeling
US10339440B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Feb 18, 2016 |
| Grant date | Jul 2, 2019 |
| Priority date | — |
| Expiry date | Mar 22, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/098
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
In some aspects, the present disclosure relates to neural language modeling. In one embodiment, a computer-implemented neural network includes a plurality of neural nodes, where each of the neural nodes has a plurality of input weights corresponding to a vector of real numbers. The neural network also includes an input neural node corresponding to a linguistic unit selected from an ordered list of a plurality of linguistic units, and an embedding layer with a plurality of embedding node partitions. Each embedding node partition includes one or more neural nodes. Each of the embedding node partitions corresponds to a position in the ordered list relative to a focus term, is configured to receive an input from an input node, and is configured to generate an output. The neural network also includes a classifier layer with a plurality of neural nodes, each configured to receive the embedding outputs from the embedding layer, and configured to generate an output corresponding to a probability that a particular linguistic unit is the focus term.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.