Efficient out of process reshuffle of streaming data
US11010171B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 30, 2019 |
| Grant date | May 18, 2021 |
| Priority date | — |
| Expiry date | May 30, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/254
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, apparatuses, and computer program products are provided for processing a stream of data. A maximum temporal divergence is established for data flushed to a data store from a plurality of upstream partitions. Each of a plurality of data flushers, each corresponding to an upstream partition, may obtain an item of data from a data producer. Each data flusher may determine whether flushing the data to the data store would exceed the maximum temporal divergence. Based at least on determining that flushing the data to the data store would not exceed the maximum temporal divergence, the data may be flushed to the data store for ingestion by a downstream partition and a data structure (e.g., a ledger) may be updated to indicate a time associated with the most recent item of data flushed to the data store.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.