Systems and methods for accelerating hessian-free optimization for deep neural networks by implicit preconditioning and sampling
US9601109B2 · kind B2 · utility
6Cited by
0References
20Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Sep 29, 2014 |
| Grant date | Mar 21, 2017 |
| Priority date | — |
| Expiry date | Apr 24, 2035 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L15/16
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method for training a deep neural network, comprises receiving and formatting speech data for the training, preconditioning a system of equations to be used for analyzing the speech data in connection with the training by using a non-fixed point quasi-Newton preconditioning scheme, and employing flexible Krylov subspace solvers in response to variations in the preconditioning scheme for different iterations of the training.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.