Prediction on top-down scenes based on action data

US11023749B2 · kind B2 · utility

Assignee

• Zoox, Inc. · US

Inventors

• Gowtham Garimella · Hayward, US
• Marin Kobilarov · Baltimore, US
• Andres Guillermo Morales Morales · San Francisco, US
• Kai Zhenyu Wang · Burnaby, CA

Key dates

• Filing date: Jul 5, 2019
• Grant date: Jun 1, 2021
• Expiry date: Aug 25, 2039

Classification

• Technology area (CPC G): Physics
• CPC primary: G06T2210/12
• WIPO field: Control
• WIPO sector: Instruments

Abstract

Techniques for determining predictions on a top-down representation of an environment based on vehicle action(s) are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle or a pedestrian). A multi-channel image representing a top-down view of the object(s) and the environment can be generated based on the sensor data, map data, and/or action data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) can be encoded in the image. Action data can represent a target lane, trajectory, etc. of the first vehicle. Multiple images can be generated representing the environment over time and input into a prediction system configured to output prediction probabilities associated with possible locations of the object(s) in the future, which may be based on the actions of the autonomous vehicle.