Regression testing question answering cognitive computing systems by applying ground truth virtual checksum techniques
US10585784B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 10, 2015 |
| Grant date | Mar 10, 2020 |
| Priority date | — |
| Expiry date | Mar 20, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N5/04
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A mechanism is provided in a data processing system for performing regression testing on a question answering system instance. The mechanism trains a machine learning model for a question answering system using a ground truth virtual checksum as part of a ground truth including domain-specific ground truth. The ground truth virtual checksum comprises a set of test questions, an answer to each test question, and a confidence level range for each answer to a corresponding test question. The mechanism runs regression test buckets across system nodes with domain-specific corpora and receiving results from the system nodes. Each system node implements a question answering system instance of the question answering system by executing in accordance with the machine learning model and by accessing domain-specific corpora. Each test bucket includes a set of questions matching a subset of questions in the ground truth virtual checksum. The mechanism identifies regressions, inconsistencies, or destabilizations in code behavior in the system nodes based on results of comparing the results to the ground truth virtual checksum and generates a report presenting the identified regressions, inconsi…
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.