Temporal fusion of multimodal data from multiple data acquisition systems to automatically recognize and classify an action
US9805255B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 29, 2016 |
| Grant date | Oct 31, 2017 |
| Priority date | — |
| Expiry date | Jan 29, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V40/20
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A multimodal sensing system includes various devices that work together to automatically classify an action. A video camera captures a sequence of digital images. At least one other sensor device captures other sensed data (e.g., motion data). The system will extract video features from the digital images so that each extracted image feature is associated with a time period. It will extract other features from the other sensed data so that each extracted other feature is associated with a time period. The system will fuse a group of the extracted video features and a group of the extracted other features to create a fused feature representation for a time period. It will then analyze the fused feature representation to identify a class, access a data store of classes and actions to identify an action that is associated with the class, and save the identified action to a memory device.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.