Memory-efficient backpropagation through time
US11256990B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 19, 2017 |
| Grant date | Feb 22, 2022 |
| Priority date | — |
| Expiry date | Apr 28, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/09
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a recurrent neural network on training sequences using backpropagation through time. In one aspect, a method includes receiving a training sequence including a respective input at each of a number of time steps; obtaining data defining an amount of memory allocated to storing forward propagation information for use during backpropagation; determining, from the number of time steps in the training sequence and from the amount of memory allocated to storing the forward propagation information, a training policy for processing the training sequence, wherein the training policy defines when to store forward propagation information during forward propagation of the training sequence; and training the recurrent neural network on the training sequence in accordance with the training policy.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.