Addressing memory limits for partition tracking among worker nodes
US11989194B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 18, 2019 |
| Grant date | May 21, 2024 |
| Priority date | — |
| Expiry date | Aug 30, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2201/86
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.