Layered scene decomposition CODEC with higher order lighting

US10986326B2 · kind B2 · utility

Inventors

• Matthew Hamilton · St. John's, CA
• Chuck Rumbolt · St. John's, CA
• Donovan Benoit · St. John's, CA
• Matthew Troke · St. John's, CA
• Robert Lockyer · St. John's, CA
• Thomas Butyn · Fletchers Lake, CA

Key dates

• Filing date: Feb 21, 2020
• Grant date: Apr 20, 2021
• Expiry date: Feb 21, 2040

Classification

• Technology area (CPC H): Electricity
• CPC primary: H04N2013/0088
• WIPO field: Audio-visual technology
• WIPO sector: Electrical engineering

Abstract

A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data including information on directions of normals is divided into a plurality of data layers of increasing depths as the distance between a given layer and the display surface increases. Data layers which are sampled using a plenoptic sampling scheme and rendered using hybrid rendering, such as perspective and oblique rendering, to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.