Actor-replacement system for videos
US11749311B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 6, 2022 |
| Grant date | Sep 5, 2023 |
| Priority date | — |
| Expiry date | Dec 6, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2021/105
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
In one aspect, an example method includes (i) estimating, using a skeletal detection model, a pose of an original actor for each of multiple frames of a video; (ii) obtaining, for each of a plurality of the estimated poses, a respective image of a replacement actor; (iii) obtaining replacement speech in the replacement actor's voice that corresponds to speech of the original actor in the video; (iv) generating, using the estimated poses, the images of the replacement actor, and the replacement speech, synthetic frames corresponding to the multiple frames of the video that depict the replacement actor in place of the original actor, with the synthetic frames including facial expressions for the replacement actor that temporally align with the replacement speech; and (iv) combining the synthetic frames and the replacement speech so as to obtain a synthetic video that replaces the original actor with the replacement actor.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.