Dynamic data partitioning for optimal resource utilization in a parallel data processing system
US9811384B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 27, 2012 |
| Grant date | Nov 7, 2017 |
| Priority date | — |
| Expiry date | Jun 27, 2032 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F9/505
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method, computer program product, and system for dynamically distributing data for parallel processing in a computing system, comprising allocating a data buffer to each of a plurality of data partitions, where each data buffer stores data to be processed by its corresponding data partition, distributing data in multiple rounds to the data buffers for processing by the data partitions, where in each round the data is distributed based on a determined data processing capacity for each data partition, and where a greater amount of data is distributed to the data partitions with higher determined processing capacities, and periodically monitoring usage of each data buffer and re-determining the determined data processing capacity of each data partition based on its corresponding data buffer usage.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.