Defending multimodal fusion models against single-source adversaries
US11893087B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 16, 2021 |
| Grant date | Feb 6, 2024 |
| Priority date | — |
| Expiry date | Jan 1, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V20/56
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A multimodal perception system for an autonomous vehicle includes a first sensor that is one of a video, RADAR, LIDAR, or ultrasound sensor, and a controller. The controller may be configured to, receive a first signal from a first sensor, a second signal from a second sensor, and a third signal from a third sensor, extract a first feature vector from the first signal, extract a second feature vector from the second signal, extract a third feature vector from the third signal, determine an odd-one-out vector from the first, second, and third feature vectors via an odd-one-out network of a machine learning network, based on inconsistent modality prediction, fuse the first, second, and third feature vectors and odd-one-out vector into a fused feature vector, output the fused feature vector, and control the autonomous vehicle based on the fused feature vector.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.