Asynchronous in-memory data checkpointing for distributed computing systems
US10853207B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 27, 2019 |
| Grant date | Dec 1, 2020 |
| Priority date | — |
| Expiry date | Nov 27, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2201/805
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are provided for implementing asynchronous checkpointing of in-memory data in a distributed computing system. For example, a method includes processing a stream of data records by an operator executing on a computing node, maintaining in a system memory, an operator state which is generated in response to the operator processing the stream of data records, and performing an asynchronous checkpointing process. The asynchronous checkpointing process includes enqueuing a checkpoint of the operator state in a first queue, wherein the first queue is maintained in the system memory, and executing a background worker thread to dequeue the checkpoint of the operator state from the first queue and store the checkpoint of the operator state in a data store. The operator continues with processing the stream of data records during the asynchronous checkpointing process.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.