Controllable video characters with natural motions extracted from real-world videos
US11017560B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 15, 2019 |
| Grant date | May 25, 2021 |
| Priority date | — |
| Expiry date | Jul 14, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/30221
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A video generation system is described that extracts one or more characters or other objects from a video, re-animates the character, and generates a new video in which the extracted characters. The system enables the extracted character(s) to be positioned and controlled within a new background scene different from the original background scene of the source video. In one example, the video generation system comprises a pose prediction neural network having a pose model trained with (i) a set of character pose training images extracted from an input video of the character and (ii) a simulated motion control signal generated from the input video. In operation, the pose prediction neural network generates, in response to a motion control input from a user, a sequence of images representing poses of a character. A frame generation neural network generates output video frames that render the character within a scene.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.