Extracting document hierarchy using a multimodal, layer-wise link prediction neural network

US12333844B2 · kind B2 · utility

Assignee

• Adobe Inc. · US

Inventors

• Vlad Ion Morariu · Potomac, US
• Puneet Mathur · College Park, US
• Rajiv Jain · Falls Church, US
• Ashutosh Mehra · Noida, IN
• Jiuxiang Gu · College Park, US
• Franck Dernoncourt · Sunnyvale, US
• Anandhavelu N · Kangeyam, IN
• Quan Tran · San Jose, US
• Verena Kaynig-Fittkau · Cambridge, US
• Nedim Lipka · Campbell, US
• Ani Nenkova · Philadelphia, US

Key dates

• Filing date: Nov 15, 2022
• Grant date: Jun 17, 2025
• Expiry date: Nov 22, 2043

Classification

• Technology area (CPC G): Physics
• CPC primary: G06V10/82
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate a digital document hierarchy comprising layers of parent-child element relationships from the visual elements. For example, for a layer of the layers, the disclosed systems determine, from the visual elements, candidate parent visual elements and child visual elements. In addition, for the layer of the layers, the disclosed systems generate, from the feature embeddings utilizing a neural network, element classifications for the candidate parent visual elements and parent-child element link probabilities for the candidate parent visual elements and the child visual elements. Moreover, for the layer, the disclosed systems select parent visual elements from the candidate parent visual elements based on the parent-child element link probabilities. Further, the disclosed systems utilize the digital document hierarchy to generate an interactive digital document from the digital document image.