Style-aware audio-driven talking head animation from a single image
US11417041B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Feb 12, 2020 |
| Grant date | Aug 16, 2022 |
| Priority date | — |
| Expiry date | May 27, 2040 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2021/105
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Embodiments of the present invention provide systems, methods, and computer storage media for generating an animation of a talking head from an input audio signal of speech and a representation (such as a static image) of a head to animate. Generally, a neural network can learn to predict a set of 3D facial landmarks that can be used to drive the animation. In some embodiments, the neural network can learn to detect different speaking styles in the input speech and account for the different speaking styles when predicting the 3D facial landmarks. Generally, template 3D facial landmarks can be identified or extracted from the input image or other representation of the head, and the template 3D facial landmarks can be used with successive windows of audio from the input speech to predict 3D facial landmarks and generate a corresponding animation with plausible 3D effects.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.