Machine learning model to fuse emergency vehicle audio and visual detection

US11620903B2 · kind B2 · utility

Assignee

• BAIDU USA LLC · US

Inventors

• Kecheng Xu · Sunnyvale, US
• Hongyi Sun · Sunnyvale, US
• Qi Luo · Sunnyvale, US
• Wei Wang · Sunnyvale, US
• Zejun Lin · Sunnyvale, US
• Wesley Reynolds · Sunnyvale, US
• Feng Q. Liu · San Jose, US
• Jiangtao Hu · Sunnyvale, US
• Jinghao Miao · Sunnyvale, US

Key dates

• Filing date: Jan 14, 2021
• Grant date: Apr 4, 2023
• Expiry date: Jul 17, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G06V2201/08
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

According to various embodiments, systems, methods, and mediums for operating an autonomous driving vehicles (ADV) are described. The embodiments use a number of machine learning models to extract features individually from audio data and visual data captured by sensors mounted on the ADV, and then to fuse these extracted features to create a concatenated feature vectors. The concatenated feature vector is provided to a multiplayer perceptron (MLP) as input to generate a detection result related to the presence of an emergency vehicle in the surrounding environment. The detection result can be used by the ADV to take appropriate actions to comply with the local traffic rules.