Self-analyzing data processing job to determine data quality issues
US9576036B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Mar 15, 2013 |
| Grant date | Feb 21, 2017 |
| Priority date | — |
| Expiry date | Mar 26, 2034 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/215
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are disclosed to determine data quality issues in data processing jobs. The data processing job is received, the data processing job specifying one or more processing steps designed based on one or more data schemas and further specifies one or more desired quality metrics to measure at the one or more processing steps. One or more state machines are provided, that are generated based on the quality metrics and on the data schemas. Input data to the data process job are processed using the one or more state machines, in order to generate output data and a set of data quality records characterizing a set of data quality issues identified during the execution of the data processing job.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.