Generating realistic facial animation from speech
US6735566B1 · kind B1 · utility
Assignee
Inventor
Key dates
| Filing date | Oct 9, 1998 |
| Grant date | May 11, 2004 |
| Priority date | — |
| Expiry date | Oct 9, 2018 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2021/105
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system for learning a mapping between time-varying signals is used to drive facial animation directly from speech, without laborious voice track analysis. The system learns dynamical models of facial and vocal action from observations of a face and the facial gestures made while speaking. Instead of depending on heuristic intermediate representations such as phonemes or visemes, the system trains hidden Markov models to obtain its own optimal representation of vocal and facial action. An entropy-minimizing training technique using an entropic prior ensures that these models contain sufficient dynamical information to synthesize realistic facial motion to accompany new vocal performances. In addition, they can make optimal use of context to handle ambiguity and relatively long-lasting facial co-articulation effects. The output of the system is a sequence of facial control parameters suitable for driving a variety of different kinds of animation ranging from warped photorealistic images to 3D cartoon characters.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.