Cloud platform based architecture for continuous deployment and execution of modular data pipelines
US12135732B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | May 24, 2023 |
| Grant date | Nov 5, 2024 |
| Priority date | — |
| Expiry date | May 24, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F8/60
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system performs delivery of a data pipeline on a cloud platform. The system receives a specification of the data pipeline that is split into smaller specifications of data pipeline units. The system identifies a target cloud platform and generates a deployment package for each data pipeline unit for the target cloud platform. The system creates a connection with the target cloud platform and uses the connection to provision computing infrastructure on the target cloud platform for the data pipeline unit according to the system configuration of the data pipeline unit. The data pipeline may be implemented as a data mesh that is a directed acyclic graph of nodes, each node representing a data pipeline unit. Different portions of the data mesh may be modified independent of each other. Partial results stored in different portions of the data mesh may be recomputed starting from different points in time.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.