Audio-based face tracking and lip syncing for natural facial animation and lip movement
US10755463B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 20, 2018 |
| Grant date | Aug 25, 2020 |
| Priority date | — |
| Expiry date | Jul 20, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2021/105
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
In one embodiment, a method includes receiving an audio signal comprising a plurality of speech units, processing the audio signal to associate each of the speech units with a corresponding lip animation, determining pitch information associated with each of the plurality of speech units, processing the pitch information of each of the plurality of speech units to associate at least one of the speech units with a facial-component animation, and presenting the audio signal with a displayed animation of a face, wherein the animation of the face displays the lip animation associated with each of the speech units and the facial-component animation associated with the at least one speech unit. The animation of the face may be displayed in real time with the audio signal. The facial component animation may include animation of the lips, eyebrows, eyelids, and other portion of the upper face.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.