Using machine-learning models to determine movements of a mouth corresponding to live speech
US10699705B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 22, 2018 |
| Grant date | Jun 30, 2020 |
| Priority date | — |
| Expiry date | Dec 22, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2021/105
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Disclosed systems and methods predict visemes from an audio sequence. A viseme-generation application accesses a first set of training data that includes a first audio sequence representing a sentence spoken by a first speaker and a sequence of visemes. Each viseme is mapped to a respective audio sample of the first audio sequence. The viseme-generation application creates a second set of training data adjusting a second audio sequence spoken by a second speaker speaking the sentence such that the second and first sequences have the same length and at least one phoneme occurs at the same time stamp in the first sequence and in the second sequence. The viseme-generation application maps the sequence of visemes to the second audio sequence and trains a viseme prediction model to predict a sequence of visemes from an audio sequence.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.