Feature pyramid warping for video frame interpolation
US12288346B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 14, 2020 |
| Grant date | Apr 29, 2025 |
| Priority date | — |
| Expiry date | Aug 29, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/20084
- WIPO fieldAudio-visual technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and storage media are described for motion estimation in video frame interpolation. Disclosed embodiments use feature pyramids as image representations for motion estimation and seamlessly integrates them into a deep neural network for frame interpolation. A feature pyramid is extracted for each of two input frames. These feature pyramids are wrapped together with the input frames to the target temporal position according to the inter-frame motion estimated via optical flow. A frame synthesis network is used to predict interpolation results from the pre-warped feature pyramids and input frames. The feature pyramid extractor and the frame synthesis network are jointly trained for the task of frame interpolation. An extensive quantitative and qualitative evaluation demonstrates that the described embodiments utilizing feature pyramids enables robust, high-quality video frame interpolation. Other embodiments may be described and/or claimed.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.