Automatic benchmarking of labeling tasks
US12189719B1 · kind B1 · utility
0Cited by
0References
20Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Jan 6, 2022 |
| Grant date | Jan 7, 2025 |
| Priority date | — |
| Expiry date | Sep 30, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V10/7784
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
One embodiment of the present invention sets forth a technique for evaluating labeled data. The technique includes selecting, from a set of labels for a data sample, a subset of the labels representing non-outliers in a distribution of values in the set of labels. The technique also includes aggregating the subset of the labels into a benchmark for the data sample. The technique further includes generating, based on a comparison between the benchmark and an additional label, a benchmark score associated with the data sample, and generating a set of performance metrics for labeling the data sample based on the benchmark score.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.