Identifying nonsense passages in a question answering system based on domain specific policy
US10585898B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 12, 2016 |
| Grant date | Mar 10, 2020 |
| Priority date | — |
| Expiry date | Sep 10, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG09B7/00
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A mechanism is provided in a data processing system for identifying nonsense passages. An annotator in a natural language processing pipeline configured to execute in the data processing system annotates an input passage in a corpus with linguistic features to form an annotated passage. A domain-specific policy is associated with a domain of the corpus. A metric counters component in the natural language processing pipeline counts a number of instances of each type of linguistic feature in the annotated passage to form a set of feature counts. The metric counters component of the natural language processing pipeline determines a value for a metric based on the set of feature counts. The metric is specified in the domain-specific policy. A comparator component of the natural language processing pipeline compares the value for the metric to a predetermined model threshold. The threshold is specified in the domain-specific policy. A filter component of the natural language processing pipeline identifies whether the input passage is a nonsense passage based on a result of the comparison.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.