Fault tolerant batch processing
US8205113B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 14, 2009 |
| Grant date | Jun 19, 2012 |
| Priority date | — |
| Expiry date | Aug 31, 2029 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F11/1471
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Among other aspects disclosed are a method and system for processing a batch of input data in a fault tolerant manner. The method includes reading a batch of input data including a plurality of records from one or more data sources and passing the batch through a dataflow graph. The dataflow graph includes two or more nodes representing components connected by links representing flows of data between the components. At least one but fewer than all of the components includes a checkpoint process for an action performed for each of multiple units of work associated with one or more of the records. The checkpoint process includes opening a checkpoint buffer stored in non-volatile memory at the start of processing for the batch. For each unit of work from the batch, if a result from performing the action for the unit of work was previously saved in the checkpoint buffer, the saved result is used to complete processing of the unit of work without performing the action again. If a result from performing the action for the unit of work is not saved in the checkpoint buffer. The action is performed to complete processing of the unit of work and the result from performing the action is save…
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.