Systems and methods for natural language processing (NLP) model robustness determination
US12073181B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 21, 2023 |
| Grant date | Aug 27, 2024 |
| Priority date | — |
| Expiry date | Apr 21, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2221/033
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems, apparatuses, methods, and computer program products are disclosed for determining robustness information for an NLP model. Modification rules, such as replacement rules and/or insertion rules, are used to generate instances of modified test data based on instances of test data that comprise words and have a syntax and a semantic meaning. The instances of test data and modified test data are provided to the NLP model and the output of the NLP model is analyzed to determine output changing instances of modified test data, which are instances of modified test data yielded output from the NLP model that is different and/or not similar to the output yielded from the NLP model for the corresponding instance of test data. Robustness information for the NLP model is determined based at least in part on the output changing instances of modified test data.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.