Managing data sparsity for neural networks

US11392829B1 · kind B1 · utility

Assignee

• NVIDIA Corporation · US

Inventors

• Jeff Pool · Santa Clara, US
• Ganesh Venkatesh · San Jose, US
• Jorge Albericio Latorre · San Jose, US
• Jack Choquette · Palo Alto, US
• Ronny Meir Krashinsky · San Francisco, US
• John Tran · Denver, US
• Feng Xie · Shanghai, CN
• Ming Y. Siu · Santa Clara, US
• Manan Patel · San Jose, US

Key dates

• Filing date: Apr 2, 2019
• Grant date: Jul 19, 2022
• Expiry date: Jun 27, 2039

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/084
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Approaches in accordance with various embodiments provide for the processing of sparse matrices for mathematical and programmatic operations. In particular, various embodiments enforce sparsity constraints for performing sparse matrix multiply-add instruction (MMA) operations. Deep neural networks can exhibit significant sparsity in the data used in operations, both in the activations and weights. The computational load can be reduced by excluding zero-valued data elements. A sparsity constraint is applied across all submatrices of a sparse matrix, providing fine-grained structured sparsity that is evenly distributed across the matrix. The matrix may then be compressed since a minimum number of elements of the matrix are known to have zero value. Matrix operations are then performed using these matrices.