Dynamic memory tuning for in-memory data analytic platforms
US10204175B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 18, 2016 |
| Grant date | Feb 12, 2019 |
| Priority date | — |
| Expiry date | May 18, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2212/621
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
At a cache manager of a directed acyclic graph-based data analytic platform, memory usage statistics are obtained from each of a plurality of monitor components on a plurality of worker nodes. The worker nodes have a plurality of tasks executing thereon, and each of the tasks has at least one distributed dataset associated therewith. Each of the worker nodes has a distributed dataset cache. At least one of the following is carried out: increasing a size of a given one of the distributed dataset caches if the memory usage statistics indicate that corresponding ones of the tasks are using too little memory; and decreasing a size of another given one of the distributed dataset caches if the memory usage statistics indicate contention between corresponding ones of the tasks and a corresponding one of the distributed datasets.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.