Calibrating confidence scores of a machine learning model trained as a natural language interface
US12430330B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Feb 9, 2023 |
| Grant date | Sep 30, 2025 |
| Priority date | — |
| Expiry date | Jan 3, 2044 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/40
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are disclosed herein for calibrating confidence scores of a machine learning model trained to translate natural language to a meaning representation language. The techniques include obtaining one or more raw beam scores generated from one or more beam levels of a decoder of a machine learning model trained to translate natural language to a logical form, where each of the one or more raw beam scores is a conditional probability of a sub-tree determined by a heuristic search algorithm of the decoder at one of the one or more beam levels, classifying, by a calibration model, a logical form output by the machine learning model as correct or incorrect based on the one or more raw beam scores, and providing the logical form with a confidence score that is determined based on the classifying of the logical form.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.