Systems and methods for training language models to reason over tables
US11868381B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Mar 29, 2021 |
| Grant date | Jan 9, 2024 |
| Priority date | — |
| Expiry date | Jan 2, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/096
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems and methods for pre-training and fine-tuning of neural-network-based language models to reason directly over tables without generating logical forms. In some examples, a language model can be pre-trained using masked-language modeling tasks synthetically generated from tables pulled from a knowledge corpus. In some examples, the language model may be further pre-trained using pairs of counterfactual statements generated from those tables, and/or one or more statements that compare selected data from those tables. The language model may then be fine-tuned using examples that include only a question, an answer, and a table, allowing fine-tuning examples to be harvested directly from existing benchmark datasets or synthetically generated.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.