Dynamic scaling for data processing streaming system
US11095522B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 21, 2019 |
| Grant date | Aug 17, 2021 |
| Priority date | — |
| Expiry date | Aug 21, 2039 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L65/60
- WIPO fieldDigital communication
- WIPO sectorElectrical engineering
Abstract
Described herein is a system and method for dynamically scaling a stream processing system (e.g., “exactly once” data stream processing system). Various parameter(s) (e.g., user-configurable capacity, real-time load metrics, and/or performance counters) can be used to dynamically scale in and/or scale out the “exactly once” stream processing system without system restart. Delay introduced by this scaling operation can be minimized by utilizing a combination of mutable process topology (which can dynamically assign certain parts of the system to a new host machine) and controllable streaming processor movement with checkpoints and the streaming protocol controlled recovery which still enforces the “exactly once” delivery metric.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.