Scalable distributed in-memory computation
US10656861B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 12, 2017 |
| Grant date | May 19, 2020 |
| Priority date | — |
| Expiry date | Feb 16, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F9/5077
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An apparatus in one embodiment comprises at least one processing device having a processor coupled to a memory. The processing device is configured to distribute in-memory computations across at least first and second nodes of respective distinct data processing clusters of a plurality of data processing clusters over at least one network, and to aggregate results of the distributed in-memory computations for delivery to a requesting client device. The data processing clusters are associated with respective distinct data zones, and the first and second nodes of the respective distinct data processing clusters are configured to perform corresponding portions of the distributed in-memory computations utilizing respective ones of first and second in-memory datasets locally accessible within their respective data zones. The in-memory computations in some embodiments illustratively comprise Spark computations, such as Spark Core batch computations. The in-memory datasets in such an arrangement may comprise respective Spark resilient distributed datasets.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.