Machine learning systems and methods for augmenting images

US10529137B1 · kind B1 · utility

Assignee

• Max-Planck-Gesellschaft zur Föderung der Wissenschaften e.V. · US

Inventors

• Michael J. Black · Lörrach, DE
• Eric Rachlin · Providence, US
• Evan Lee · New York, US
• Nicolas Heron · New York, US
• Matthew M. Loper · Providence, US
• Alexander W. Weiss · Linden, DE
• David J. Smith · Columbia, US

Key dates

• Filing date: Nov 29, 2017
• Grant date: Jan 7, 2020
• Expiry date: Jan 6, 2038

Classification

• Technology area (CPC G): Physics
• CPC primary: G06T2219/2021
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Disclosed is a method including receiving visual input comprising a human within a scene, detecting a pose associated with the human using a trained machine learning model that detects human poses to yield a first output, estimating a shape (and optionally a motion) associated with the human using a trained machine learning model associated that detects shape (and optionally motion) to yield a second output, recognizing the scene associated with the visual input using a trained convolutional neural network which determines information about the human and other objects in the scene to yield a third output, and augmenting reality within the scene by leveraging one or more of the first output, the second output, and the third output to place 2D and/or 3D graphics in the scene.