Generating natural language representations of mental content from functional brain images

US10856815B2 · kind B2 · utility

Assignee

• Siemens Medical Solutions USA, Inc. · US

Inventors

• Francisco Pereira · Princeton, US
• Bin Lou · Princeton Junction, US
• Angeliki Lazaridou · Artemida, GR

Key dates

• Filing date: Oct 21, 2016
• Grant date: Dec 8, 2020
• Expiry date: Jun 20, 2037

Classification

• Technology area (CPC A): Human Necessities
• CPC primary: A61B2576/026
• WIPO field: Medical technology
• WIPO sector: Instruments

Abstract

By way of introduction, the present embodiments described below include apparatuses and methods for generating natural language representations of mental content from functional brain images. Given functional imaging data acquired while a subject reads a text passage, a reconstruction of the text passage is produced. Linguistic semantic vector representations are assigned (1301) to words, phrases or sentences to be used as training stimuli. Basis learning is performed (1305), using brain imaging data acquired (1303) when a subject is exposed to the training stimuli and the corresponding semantic vectors for training stimuli, to learn an image basis directly. Semantic vector decoding (1309) is performed with functional brain imaging data for test stimuli and using the image basis to generate a semantic vector representing the test imaging stimuli. Text generation (1311) is then performed using the decoded semantic vector representing the test imaging stimuli.