Robotic control using value distributions

US11571809B1 · kind B1 · utility

Assignee

• X Development LLC · US

Inventors

• Cristian Bodnar · Palo Alto, US
• Adrian Li · San Francisco, US
• Karol Hausman · San Francisco, US
• Peter Pastor Sampedro · Oakland, US
• Mrinal Kalakrishnan · Palo Alto, US

Key dates

• Filing date: Sep 11, 2020
• Grant date: Feb 7, 2023
• Expiry date: May 11, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G05B2219/40499
• WIPO field: Handling
• WIPO sector: Mechanical engineering

Abstract

Techniques are described herein for robotic control using value distributions. In various implementations, as part of performing a robotic task, state data associated with the robot in an environment may be generated based at least in part on vision data captured by a vision component of the robot. A plurality of candidate actions may be sampled, e.g., from continuous action space. A trained critic neural network model that represents a learned value function may be used to process a plurality of state-action pairs to generate a corresponding plurality of value distributions. Each state-action pair may include the state data and one of the plurality of sampled candidate actions. The state-action pair corresponding to the value distribution that satisfies one or more criteria may be selected from the plurality of state-action pairs. The robot may then be controlled to implement the sampled candidate action of the selected state-action pair.