Credibility of text analysis engine performance evaluation by rating reference content
US8214734B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 9, 2008 |
| Grant date | Jul 3, 2012 |
| Priority date | — |
| Expiry date | Mar 27, 2031 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/355
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Evaluating the performance of a text analysis engine is provided. A plurality of pre-annotated reference documents and a set of annotation types associated with the pre-annotated reference documents are received. Annotation contexts of reference annotations in the plurality of pre-annotated reference documents are analyzed using the set of annotation types. Similar annotation contexts are identified between the reference annotations and the set of annotation types. Responsive to identifying the similar annotation contexts, the similar annotation contexts are clustered thereby forming a plurality of reference annotation clusters. A set of reference content heterogeneity scores are computed based on the number of reference annotation clusters for each annotation type in the set of annotation types. An integral reference content rate for the set of annotation types is then computed and output to a user.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.