Method and system for generating 2D animated lip images synchronizing to an audio signal
US11887238B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 18, 2021 |
| Grant date | Jan 30, 2024 |
| Priority date | — |
| Expiry date | Jun 21, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2021/105
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method and system for generating 2D animated lip images synchronizing to an audio signal for an unseen subject. The system receives an audio signal and a target lip image of an unseen target subject as inputs from a user and processes these inputs to extract a plurality of high dimensional audio image features. The lip generator system is meta-trained with training dataset which consists of large variety of subjects' ethnicity and vocabulary. The meta-trained model generates realistic animation for previously unseen face and unseen audio when finetuned with only a few-shot samples for a predefined interval of time. Additionally, the method protects intrinsic features of the unseen target subject.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.