Using multimodal model consistency to detect adversarial attacks
US11675896B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 9, 2020 |
| Grant date | Jun 13, 2023 |
| Priority date | — |
| Expiry date | Feb 20, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/048
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method, apparatus and computer program product to defend learning models that are vulnerable to adversarial example attack. It is assumed that data (a “dataset”) is available in multiple modalities (e.g., text and images, audio and images in video, etc.). The defense approach herein is premised on the recognition that the correlations between the different modalities for the same entity can be exploited to defend against such attacks, as it is not realistic for an adversary to attack multiple modalities. To this end, according to this technique, adversarial samples are identified and rejected if the features from one (the attacked) modality are determined to be sufficiently far away from those of another un-attacked modality for the same entity. In other words, the approach herein leverages the consistency between multiple modalities in the data to defend against adversarial attacks on one modality.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.