Training with adaptive runtime and precision profiling

US11017291B2 · kind B2 · utility

Assignee

• Intel Corporation · US

Inventors

• Brian T. Lewis · Palo Alto, US
• Rajkishore Barik · Santa Clara, US
• Murali Sundaresan · Sunnyvale, US
• Leonard Truong · Santa Clara, US
• Feng Chen · Shanghai, CN
• Xiaoming Chen · Xi'an, CN
• Mike B. Macpherson · Portland, US

Key dates

• Filing date: Apr 28, 2017
• Grant date: May 25, 2021
• Expiry date: Sep 9, 2039

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/098
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

A mechanism is described for facilitating efficient training of neural networks at computing devices. A method of embodiments, as described herein, includes detecting one or more inputs for training of a neural network, and introducing randomness in floating point (FP) numbers to prevent overtraining of the neural network, where introducing randomness includes replacing less-significant low-order bits of operand and result values with new low-order bits during the training of the neural network.