Lossless tiling in convolution networks—weight gradient calculation

US11232360B1 · kind B1 · utility

Assignee

• SambaNova Systems, Inc. · US

Inventors

• Tejas Nagendra Babu Nama · Sunnyvale, US
• Ruddhi Chaphekar · Santa Clara, US
• Ram Sivaramakrishnan · San Jose, US
• Raghu Prabhakar · San Jose, US
• Sumti Jairath · Santa Clara, US
• Junjue Wang · San Mateo, US
• Kaizhao Liang · Palo Alto, US
• Adi Fuchs · Princeton Colonial Park, US
• Matheen Musaddiq · Austin, US
• Arvind Krishna Sujeeth · San Francisco, US

Key dates

• Filing date: Mar 29, 2021
• Grant date: Jan 25, 2022
• Expiry date: Mar 29, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/09
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Disclosed is a data processing system that includes compile time logic configured to process a processing graph to generate a modified processing graph, which includes a plurality of forward processing nodes of a forward pass and a plurality of backward processing nodes of a backward pass. The data processing system also includes runtime logic configured with the compile time logic to execute the modified processing graph to generate, at a backward processing node of the plurality of backward processing nodes, a plurality of partial weight gradients, based on processing a corresponding plurality of gradient tiles of a gradient tensor, and generate, based on the plurality of partial weight gradients, a final weight gradient corresponding to the gradient tensor.