Scene-aware video encoder system and method
US11582485B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Feb 7, 2022 |
| Grant date | Feb 14, 2023 |
| Priority date | — |
| Expiry date | Feb 7, 2042 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04N21/251
- WIPO fieldAudio-visual technology
- WIPO sectorElectrical engineering
Abstract
Embodiments of the present disclosure discloses a scene-aware video encoder system. The scene-aware encoder system transforms a sequence of video frames of a video of a scene into a spatio-temporal scene graph. The spatio-temporal scene graph includes nodes representing one or multiple static and dynamic objects in the scene. Each node of the spatio-temporal scene graph describes an appearance, a location, and/or a motion of each of the objects (static and dynamic objects) at different time instances. The nodes of the spatio-temporal scene graph are embedded into a latent space using a spatio-temporal transformer encoding different combinations of different nodes of the spatio-temporal scene graph corresponding to different spatio-temporal volumes of the scene. Each node of the different nodes encoded in each of the combinations is weighted with an attention score determined as a function of similarities of spatio-temporal locations of the different nodes in the combination.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.