Systems and methods for natural language processing (NLP) model robustness determination
US11669687B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 12, 2020 |
| Grant date | Jun 6, 2023 |
| Priority date | — |
| Expiry date | Sep 22, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2221/033
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems, apparatuses, methods, and computer program products are disclosed for determining robustness information for an NLP model. Modification rules, such as replacement rules and/or insertion rules, are used to generate instances of modified test data based on instances of test data that comprise words and have a syntax and a semantic meaning. The instances of test data and modified test data are provided to the NLP model and the output of the NLP model is analyzed to determine output changing instances of modified test data, which are instances of modified test data yielded output from the NLP model that is different and/or not similar to the output yielded from the NLP model for the corresponding instance of test data. Robustness information for the NLP model is determined based at least in part on the output changing instances of modified test data. White and/or black box attacks may be performed.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.