Framework of proactive and/or reactive strategies for improving labeling consistency and efficiency
US12079648B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 28, 2017 |
| Grant date | Sep 3, 2024 |
| Priority date | — |
| Expiry date | Dec 20, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/285
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Performance of a computer implementing a machine learning system is improved by providing, via a graphical user interface, to an annotator, unlabeled corpus data to be labeled; obtaining, via the interface, labels for the unlabeled corpus data; and detecting, with a consistency calculation routine, concurrent with the labeling, internal inconsistency and/or external inconsistency in the labeling. Responsive to the detection, intervene in the labeling with a reactive intervention subsystem until the inconsistency is addressed. The labeling is completed subsequent to the intervention; the system is trained to provide a trained machine learning system, based on results of the completed labeling; and classification of new data is carried out with the trained system. Proactive intervention schemes are also provided.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.