Patent · US Active

Memory efficient scalable deep learning with model parallelization

US10474951B2 · kind B2 · utility

1Cited by

3References

13Claims

0Family size

Assignee

NEC CORPORATION · JP

Inventors

Renqiang Min · Princeton, US
Huahua Wang · Minneapolis, US
Asim Kadav · Mountain View, US

Key dates

Filing date	Sep 21, 2016
Grant date	Nov 12, 2019
Priority date	—
Expiry date	Mar 9, 2038

Classification

Technology area (CPC Y)Emerging Cross-Sectional Technologies
CPC primaryY04S10/50
WIPO fieldComputer technology
WIPO sectorElectrical engineering

Abstract

Methods and systems for training a neural network include sampling multiple local sub-networks from a global neural network. The local sub-networks include a subset of neurons from each layer of the global neural network. The plurality of local sub-networks are trained at respective local processing devices to produce trained local parameters. The trained local parameters from each local sub-network are averaged to produce trained global parameters.

Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.