Free viewpoint video generation and interaction method based on deep convolutional neural network
US12225176B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 28, 2020 |
| Grant date | Feb 11, 2025 |
| Priority date | — |
| Expiry date | Dec 1, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/20081
- WIPO fieldAudio-visual technology
- WIPO sectorElectrical engineering
Abstract
A Free Viewpoint Video (FVV) generation and interaction method based on a deep Convolutional Neural Network (CNN) includes the steps of: acquiring multi-viewpoint data of a target scene by a synchronous shooting system with a multi-camera array arranged accordingly to obtain groups of synchronous video frame sequences from a plurality of viewpoints, and rectifying baselines of the sequences at pixel level in batches; extracting, by encoding and decoding network structures, features of each group of viewpoint images input into a designed and trained deep CNN model, to obtain deep feature information of the scene, and combining the information with the input images to generate a virtual viewpoint image between each group of adjacent physical viewpoints at every moment; and synthesizing all viewpoints into frames of the FVV based on time and spatial position of viewpoints by stitching matrices. The method does not require camera rectification and depth image calculation.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.