System and method for automatically capturing and recording lineage data for big data records
US10037329B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 18, 2015 |
| Grant date | Jul 31, 2018 |
| Priority date | — |
| Expiry date | Oct 5, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/2471
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system may read source data corresponding to a source variable and apply a transformation to the source variable to generate an output variable. The transformation may include logic, and the output variable may be configured for ingestion into a big data storage format. The system may record lineage data of the output variable that comprises the transformation and/or the source variable. The system may also receive a request to generate a requested output variable. The requested output variable may be generated from a second transformation that is the same as the first transformation. The system may thus match the first transformation to the second transformation using the lineage data. In response to matching the first transformation to the second transformation, the system may deny the request. The original output variable may be returned in response to the matching the first transformation to the second transformation.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.