Skeleton-based action recognition using bi-directional spatial-temporal transformer
US11854305B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 9, 2021 |
| Grant date | Dec 26, 2023 |
| Priority date | — |
| Expiry date | Dec 29, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V2201/07
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A bi-directional spatial-temporal transformer neural network (BDSTT) is trained to predict original coordinates of a skeletal joint in a specific frame through relative relationships of the skeletal joint to other joints and to the state of the skeletal joint in other frames. Obtain a plurality of frames comprising coordinates of the skeletal joint and coordinates of other joints. Produce a spatially masked frame by masking the original coordinates of the skeletal joint. Provide the specific frame, the spatially masked frame, and at least one more frame to a coordinate prediction head of the BDSTT. Obtain, from the coordinate prediction head, a prediction of coordinates for the skeletal joint. Adjust parameters of the BDSTT until a mean-squared error, between the prediction of coordinates for the skeletal joint and the original coordinates of the skeletal joint, converges.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.