6D pose and shape estimation method

US12340535B2 · kind B2 · utility

Assignee

• TOYOTA JIDOSHA KABUSHIKI KAISHA · JP

Inventors

• Sven Meier · Brussels, BE
• Norimasa Kobori · Nagoya, JP
• Luca Minciullo · Brussels, BE
• Kei Yoshikawa · Nagoya, JP
• Fabian Manhardt · Garching bei München, DE
• Manuel Nickel · Munich, DE
• Nassir Navab · Munich, DE

Key dates

• Filing date: Feb 21, 2020
• Grant date: Jun 24, 2025
• Expiry date: Mar 24, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G06T2207/30252
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

A computer-implemented method of estimating a 6D pose and shape of one or more objects from a 2D image, comprises the steps of: detecting, within the 2D image, one or more 2D regions of interest, each 2D region of interest containing a corresponding object among the one of more objects; cropping out a corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; concatenating the corresponding pixel value array, coordinate tensor, and feature map for each 2D region of interest; and inferring, for each 2D region of interest, a 4D quaternion describing a rotation of the corresponding object in the 3D rotation group, a 2D centroid, which is a projection of a 3D translation of the corresponding object onto a plane of the 2D image given a camera matrix associated to the 2D image, a distance from a viewpoint of the 2D image to the corresponding object, a size, and a class-specific latent shape vector of the corresponding object.