Systems and methods for deep localization and segmentation with a 3D semantic map
US11030525B2 · kind B2 · utility
Assignees
Inventors
Key dates
| Filing date | Feb 9, 2018 |
| Grant date | Jun 8, 2021 |
| Priority date | — |
| Expiry date | Apr 3, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/30244
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Presented are deep learning-based systems and methods for fusing sensor data, such as camera images, motion sensors (GPS/IMU), and a 3D semantic map to achieve robustness, real-time performance, and accuracy of camera localization and scene parsing useful for applications such as robotic navigation and augment reality. In embodiments, a unified framework accomplishes this by jointly using camera poses and scene semantics in training and testing. To evaluate the presented methods and systems, embodiments use a novel dataset that is created from real scenes and comprises dense 3D semantically labeled point clouds, ground truth camera poses obtained from high-accuracy motion sensors, and pixel-level semantic labels of video camera images. As demonstrated by experimental results, the presented systems and methods are mutually beneficial for both camera poses and scene semantics.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.