System and method for continuous multimodal speech and gesture interaction
US9152376B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 1, 2011 |
| Grant date | Oct 6, 2015 |
| Priority date | — |
| Expiry date | Jun 14, 2032 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L2015/223
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Disclosed herein are systems, methods, and non-transitory computer-readable storage media for processing multimodal input. A system configured to practice the method continuously monitors an audio stream associated with a gesture input stream, and detects a speech event in the audio stream. Then the system identifies a temporal window associated with a time of the speech event, and analyzes data from the gesture input stream within the temporal window to identify a gesture event. The system processes the speech event and the gesture event to produce a multimodal command. The gesture in the gesture input stream can be directed to a display, but is remote from the display. The system can analyze the data from the gesture input stream by calculating an average of gesture coordinates within the temporal window.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.