Method and system for end-to-end learning of control commands for autonomous vehicle

US11016495B2 · kind B2 · utility

Assignees

• GM GLOBAL TECHNOLOGY OPERATIONS LLC · US
• CARNEGIE MELLON UNIVERSITY · US

Inventors

• Praveen Palanisamy · Sterling Heights, US
• Upali P. Mudalige · Oakland Charter Township, US
• Yilun Chen · San Diego, US
• John M. Dolan · Woodbury, US
• Katharina Muelling · Pittsburgh, US

Key dates

• Filing date: Nov 5, 2018
• Grant date: May 25, 2021
• Expiry date: Jul 2, 2039

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/048
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Systems and methods are provided for end-to-end learning of commands for controlling an autonomous vehicle. A pre-processor pre-processes image data acquired by sensors at a current time step (CTS) to generate pre-processed image data that is concatenated with additional input(s) (e.g., a segmentation map and/or optical flow map) to generate a dynamic scene output. A convolutional neural network (CNN) processes the dynamic scene output to generate a feature map that includes extracted spatial features that are concatenated with vehicle kinematics to generate a spatial context feature vector. An LSTM network processes, during the (CTS), the spatial context feature vector at the (CTS) and one or more previous LSTM outputs at corresponding previous time steps to generate an encoded temporal context vector at the (CTS). The fully connected layer processes the encoded temporal context vector to learn control command(s) (e.g., steering angle, acceleration rate and/or a brake rate control commands).