Self-correcting pipeline flows for schema drift
US12147395B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 17, 2019 |
| Grant date | Nov 19, 2024 |
| Priority date | — |
| Expiry date | Oct 13, 2040 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/258
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques describes herein updating pipeline flows in within data systems to maintain data integrity and consistency without manual curation. In certain embodiments, data integration system may detect and/or receive indications of a schema change within a source system of the data integration system. One or more data objects affected by the schema change may be identified, and a set of pipeline rules may be invoked for each of the affected schema changes. The pipeline rules may define a single transformation or a multi-step transformation process by which the data in the source system is provided to one or more target systems. After applying the pipeline rules to the updated source schema, the data received from the source system may be processed using the updated pipeline rules, transformed, and transmitted to the target system(s) to maintain the data integrity of the system.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.