Data-efficient reinforcement learning for continuous control tasks

US10664725B2 · kind B2 · utility

Assignee

• DeepMind Technologies Limited · GB

Inventors

• Martin Riedmiller · Balgheim, DE
• Roland Hafner · Spaichingen, DE
• Mel Vecerik · London, GB
• Timothy Paul Lillicrap · London, GB
• Thomas Lampe · London, GB
• Ivaylo Popov · Hanau, DE
• Gabriel Barth-Maron · London, GB
• Nicolas Manfred Otto Heess · London, GB

Key dates

• Filing date: Jul 31, 2019
• Grant date: May 26, 2020
• Expiry date: Jul 31, 2039

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/098
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.