Eye gaze for spoken language understanding in multi-modal conversational interactions
US10317992B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Sep 25, 2014 |
| Grant date | Jun 11, 2019 |
| Priority date | — |
| Expiry date | Oct 17, 2034 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V40/20
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.