Mixed intelligence data labeling system for machine learning
US10867215B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 11, 2019 |
| Grant date | Dec 15, 2020 |
| Priority date | — |
| Expiry date | Jun 14, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F18/22
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method of hybrid data labeling for machine learning, including receiving multiple unlabeled objects forming an unlabeled data set, pre-labeling the unlabeled data set by a machine learning system to output a pending label data pool, bifurcating the pending label data pool by the machine learning system into high and low confidence sets, dispatching the high confidence set to a machine labeler, dispatching the low confidence set to a human labeler, merging the label sets to return a pre-review label data pool, determining a difference between the pending label data pool and the pre-review label data pool, review labeling the data objects, if the determined difference of the data objects is greater than a predefined error threshold and storing the data objects to a reviewed pool if the determined difference of the data objects is less than and equal to the predefined error threshold.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.