Single-channel and multi-channel source separation enhanced by lip motion
US11823699B2 · kind B2 · utility
Assignee
Inventor
Key dates
| Filing date | May 23, 2022 |
| Grant date | Nov 21, 2023 |
| Priority date | — |
| Expiry date | May 23, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/30201
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods and systems are provided for implementing source separation techniques, and more specifically performing source separation on mixed source single-channel and multi-channel audio signals enhanced by inputting lip motion information from captured image data, including selecting a target speaker facial image from a plurality of facial images captured over a period of interest; computing a motion vector based on facial features of the target speaker facial image; and separating, based on at least the motion vector, audio corresponding to a constituent source from a mixed source audio signal captured over the period of interest. The mixed source audio signal may be captured from single-channel or multi-channel audio capture devices. Separating audio from the audio signal may be performed by a fusion learning model comprising a plurality of learning sub-models. Separating the audio from the audio signal may be performed by a blind source separation (“BSS”) learning model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.