Sequence generation using neural networks with continuous outputs
US10691901B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 12, 2019 |
| Grant date | Jun 23, 2020 |
| Priority date | — |
| Expiry date | Jul 12, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2209/547
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A machine learning system including a continuous embedding output layer is provided. Whereas traditional machine language translation or generation models utilize an output layer that include an single output for each word in the output vocabulary V, the present machine learning system includes a continuous embedding output layer that stores continuous vectors mapped to an m-dimensional vector space, where m is less than V. Accordingly, the present machine learning system processes an input string to produce an output vector and then searches for the continuous vector within the vector space that most closely corresponding to the output vector via, for example, a k-nearest neighbor algorithm. The system then outputs the output string corresponding to the determined continuous vector. The present system can be trained utilizing a cosine-based loss function.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.