Generating gesture reenactment video from video motion graphs using machine learning
US12347135B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 14, 2022 |
| Grant date | Jul 1, 2025 |
| Priority date | — |
| Expiry date | Nov 1, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/30241
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Embodiments are disclosed for generating a gesture reenactment video sequence corresponding to a target audio sequence using a trained network based on a video motion graph generated from a reference speech video. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input including a reference speech video and generating a video motion graph representing the reference speech video, where each node is associated with a frame of the reference video sequence and reference audio features of the reference audio sequence. The disclosed systems and methods further comprise receiving a second input including a target audio sequence, generating target audio features, identifying a node path through the video motion graph based on the target audio features and the reference audio features, and generating an output media sequence based on the identified node path through the video motion graph paired with the target audio sequence.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.