Attention shifting of a robot in a group conversation using audio-visual perception based speaker localization
US11127401B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 22, 2020 |
| Grant date | Sep 21, 2021 |
| Priority date | — |
| Expiry date | Jul 22, 2040 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2015/0635
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
This disclosure relates to attention shifting of a robot in a group conversation with two or more attendees, wherein at least one of them is a speaker. State of the art has dealt with several aspects of Human-Robot Interaction (HRI) including responding to a source of sound at a time, addressing a fixed viewing area or determining who is the speaker based on eye gaze direction. However, attention shifting to make the conversation human-like is a challenge. The present disclosure uses audio-visual perception for speaker localization. Only qualified direction of arrivals (DOAs) are used for the audio perception. Further the audio perception is complimented by visual perception employing real time face detection and lip movement detection. Use of HRI rules, clustering of the DOAs, dynamic adjustment of rotation of the robot and a dynamically updated knowledge repository enriches the robot with intelligence to shift attention with minimum human intervention.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.