Learned evaluation model for grading quality of natural language generation outputs
US11704506B2 · kind B2 · utility
3Cited by
0References
20Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Dec 4, 2020 |
| Grant date | Jul 18, 2023 |
| Priority date | — |
| Expiry date | Aug 13, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/284
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators. In some cases, following fine-tuning, the learned evaluation model may be distilled into a student model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.