Defending machine learning systems from adversarial attacks

US11893111B2 · kind B2 · utility

Assignee

• Harman International Industries, Incorporated · US

Inventors

• Srinivas Kruthiveti Subrahmanyeswara Sai · Kanchinakote, IN
• Aashish Kumar · Survanagiri, IN
• Alexander Kreines · Jerusalem, IL
• George Jose · Bengaluru, IN
• Sambuddha Saha · Ranchi, IN
• Nir Morgulis · Petah Tikva, IL
• Shachar Mendelowitz · Nes Ziona, IL

Key dates

• Filing date: Nov 26, 2019
• Grant date: Feb 6, 2024
• Expiry date: Mar 26, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G06F2221/034
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Techniques are disclosed for detecting adversarial attacks. A machine learning (ML) system processes the input into and output of a ML model using an adversarial detection module that does not include a direct external interface. The adversarial detection module includes a detection model that generates a score indicative of whether the input is adversarial using, e.g., a neural fingerprinting technique or a comparison of features extracted by a surrogate ML model to an expected feature distribution for the output of the ML model. In turn, the adversarial score is compared to a predefined threshold for raising an adversarial flag. Appropriate remedial measures, such as notifying a user, may be taken when the adversarial score satisfies the threshold and raises the adversarial flag.