Joint learning of geometry and motion with three-dimensional holistic understanding
US10970856B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 27, 2018 |
| Grant date | Apr 6, 2021 |
| Priority date | — |
| Expiry date | May 7, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/30244
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Described herein are systems and methods for jointly learning geometry and motion with three-dimensional holistic understanding. In embodiments, such approaches enforce the inherent geometrical consistency during the learning process, yielding improved results for both tasks. In embodiments, three parallel networks are adopted to predict the camera motion (e.g., MotionNet), dense depth map (e.g., DepthNet), and per-pixel optical flow between consecutive frames (e.g., FlowNet), respectively. The information of 2D flow, camera pose, and depth maps, are fed into a holistic 3D motion parser (HMP) to disentangle and recover per-pixel 3D motion of both rigid background and moving objects. Various loss terms are formulated to jointly supervise the three networks. Embodiments of an efficient iterative training strategy are disclosed for better performance and more efficient convergence. Performance on depth estimation, optical flow estimation, odometry, moving object segmentation, and scene flow estimation demonstrates the effectiveness of the disclosed systems and methods.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.