Duplicate detection and replay to ensure exactly-once delivery in a streaming pipeline
US12061608B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 22, 2020 |
| Grant date | Aug 13, 2024 |
| Priority date | — |
| Expiry date | Jan 5, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F9/542
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising processing a plurality of events using a stream processor and executing a deduplication process on the plurality of events using the stream processor. The plurality of events is outputted to a streaming queue and a close of books (COB) of a data transport is detected. Then, an audit process is initiated in response to detecting the COB signal, the audit process comprising comparing a set of raw events to a set of events in the streaming queue to identify a set of missing events, and replaying a set of missing events through the stream processor.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.