Systems and methods for labeling source data using confidence labels
US9704106B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 27, 2016 |
| Grant date | Jul 11, 2017 |
| Priority date | — |
| Expiry date | May 27, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N20/00
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems and methods for the annotation of source data using confidence labels in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining confidence labels for crowdsourced annotations includes obtaining a set of source data, obtaining a set of training data representative of the set of source data, determining the ground truth for each piece of training data, obtaining a set of training data annotations including a confidence label, measuring annotator accuracy data for at least one piece of training data, and automatically generating a set of confidence labels for the set of unlabeled data based on the measured annotator accuracy data and the set of annotator labels used.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.