Patent · US Active

Accelerating deep neural network training with inconsistent stochastic gradient descent

US10572800B2 · kind B2 · utility

5Cited by

0References

2Claims

0Family size

Assignee

NEC CORPORATION · JP

Inventors

Linnan Wang · Piscataway, US
Yi Yang · Princeton, US
Renqiang Min · Princeton, US
Srimat Chakradhar · Manalapan, US

Key dates

Filing date	Feb 2, 2017
Grant date	Feb 25, 2020
Priority date	—
Expiry date	Jun 30, 2038

Classification

Technology area (CPC G)Physics
CPC primaryG06N3/098
WIPO fieldComputer technology
WIPO sectorElectrical engineering

Abstract

Aspects of the present disclosure describe techniques for training a convolutional neural network using an inconsistent stochastic gradient descent (ISGD) algorithm. Training effort for training batches used by the ISGD algorithm are dynamically adjusted according to a determined loss for a given training batch which are classified into two sub states—well-trained or under-trained. The ISGD algorithm provides more iterations for under-trained batches while reducing iterations for well-trained ones.

Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.