Dynamic tuning of memory in MapReduce systems
US9582189B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 25, 2014 |
| Grant date | Feb 28, 2017 |
| Priority date | — |
| Expiry date | Dec 30, 2034 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F9/5016
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and computer program products for dynamic tuning of memory in MapReduce systems are provided herein. A method includes analyzing (i) memory usage of a first sub-set of multiple tasks associated with a MapReduce job and (ii) an amount of data utilized across the first sub-set of the multiple tasks; determining a memory size to be allocated to the first sub-set of the multiple tasks based on said analyzing, wherein said memory size minimizes a cost function related to said memory usage and said amount of data utilized; performing a task-wise performance comparison among a second sub-set of the multiple tasks associated with the MapReduce job using the determined memory size to be allocated to the first sub-set of the multiple tasks to generate a set of memory allocation results; and dynamically applying the set of memory allocation results to one or more additional tasks associated with the MapReduce job.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.