Listener animation

US12254548B1 · kind B1 · utility

Assignee

• AMAZON TECHNOLOGIES, INC. · US

Inventors

• Gourav Datta · Los Angeles, US
• Vivek Yadav · Austin, US
• Yue Wu · Mountain View, US
• Ayush Jaiswal · Los Angeles, US
• Rajiv Reddy · Bellevue, US
• Prateek Singhal · Mountain View, US
• Karthik Ramakrishnan · Burlington, CA
• Premkumar Natarajan · Belmont, US

Key dates

• Filing date: Dec 16, 2022
• Grant date: Mar 18, 2025
• Expiry date: Jun 15, 2043

Classification

• Technology area (CPC G): Physics
• CPC primary: G06T2207/30201
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

A system configured to perform style-aware listener animation. By representing different listening styles (e.g., facial expressions) using an embedding space, a single model can be trained to generate unique facial animations for a number of distinct listeners. Thus, individual listening styles can be associated with a listener identifier, enabling the system to (i) animate a plurality of different listeners with unique nonverbal behavior and/or (ii) select a particular listener identifier or desired type of listener style with which to animate. This enables the model to be generalized to new listeners to generate additional listener facial responses without needing training data for each new listener. The model may process a listener representation style or listener identifier, along with input data corresponding to a speaker talking, to generate unique facial animation responsive to the speech.