Detecting poisoning attacks on neural networks by activation clustering

US11188789B2 · kind B2 · utility

Assignee

• International Business Machines Corporation · US

Inventors

• Bryant Chen · San Jose, US
• Wilka Carvalho · Los Angeles, US
• Heiko H. Ludwig · San Francisco, US
• Ian M. Molloy · Chappaqua, US
• Taesung Lee · White Plains, US
• Jialong Zhang · White Plains, US
• Benjamin J. Edwards · Palm Harbor, US

Key dates

• Filing date: Aug 7, 2018
• Grant date: Nov 30, 2021
• Expiry date: Oct 1, 2040

Classification

• Technology area (CPC G): Physics
• CPC primary: G06V30/10
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

One embodiment provides a method comprising receiving a training set comprising a plurality of data points, where a neural network is trained as a classifier based on the training set. The method further comprises, for each data point of the training set, classifying the data point with one of a plurality of classification labels using the trained neural network, and recording neuronal activations of a portion of the trained neural network in response to the data point. The method further comprises, for each classification label that a portion of the training set has been classified with, clustering a portion of all recorded neuronal activations that are in response to the portion of the training set, and detecting one or more poisonous data points in the portion of the training set based on the clustering.