Quantifying vulnerabilities of deep learning computing systems to adversarial perturbations

US11227215B2 · kind B2 · utility

Assignee

• International Business Machines Corporation · US

Inventors

• Sijia Liu · Beijing, CN
• Quanfu Fan · Somerville, US
• Chuang Gan · Cambridge, US
• Dakuo Wang · Cambridge, US

Key dates

• Filing date: Mar 8, 2019
• Grant date: Jan 18, 2022
• Expiry date: Mar 9, 2040

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/045
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Mechanisms are provided for generating an adversarial perturbation attack sensitivity (APAS) visualization. The mechanisms receive a natural input dataset and a corresponding adversarial attack input dataset, where the adversarial attack input dataset comprises perturbations intended to cause a misclassification by a computer model. The mechanisms determine a sensitivity measure of the computer model to the perturbations in the adversarial attack input dataset based on a processing of the natural input dataset and corresponding adversarial attack input dataset by the computer model. The mechanisms generate a classification activation map (CAM) for the computer model based on results of the processing and a sensitivity overlay based on the sensitivity measure. The sensitivity overlay graphically represents different classifications of perturbation sensitivities. The mechanisms apply the sensitivity overlay to the CAM to generate and output a graphical visualization output of the computer model sensitivity to perturbations of adversarial attacks.