Deep novel view and lighting synthesis from sparse images
US10950037B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 12, 2019 |
| Grant date | Mar 16, 2021 |
| Priority date | — |
| Expiry date | Jul 12, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/20084
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Embodiments are generally directed to generating novel images of an object having a novel viewpoint and a novel lighting direction based on sparse images of the object. A neural network is trained with training images rendered from a 3D model. Utilizing the 3D model, training images, ground truth predictive images from particular viewpoint(s), and ground truth predictive depth maps of the ground truth predictive images, can be easily generated and fed back through the neural network for training. Once trained, the neural network can receive a sparse plurality of images of an object, a novel viewpoint, and a novel lighting direction. The neural network can generate a plane sweep volume based on the sparse plurality of images, and calculate depth probabilities for each pixel in the plane sweep volume. A predictive output image of the object, having the novel viewpoint and novel lighting direction, can be generated and output.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.