Data processing pipeline error recovery
US11630730B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 29, 2021 |
| Grant date | Apr 18, 2023 |
| Priority date | — |
| Expiry date | Oct 29, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2209/505
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are disclosed for executing a data processing pipeline. The techniques may include receiving a job at a data pipeline queue, setting up one or more distributed processing environments, and allocating the job to one of the distributed processing environments. The techniques may further include receiving the allocated job at a job queue within the distributed processing environment, increasing a priority level of the job, and executing the job within the distributed processing environment. The techniques can further include providing a retry pipeline at the data processing pipeline, and re-executing the job at a stage following a failure of at least one of its components. The techniques may decrement the retry budget as the job is re-executed.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.