Optimizing data partitioning for data-parallel computing
US9235396B2 · kind B2 · utility
5Cited by
3References
20Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Dec 13, 2011 |
| Grant date | Jan 12, 2016 |
| Priority date | — |
| Expiry date | Mar 3, 2033 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F8/453
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A data partitioning plan is automatically generated that—given a data-parallel program and a large input dataset, and without having to first run the program on the input dataset—substantially optimizes performance of the distributed execution system that explicitly measures and infers various properties of both data and computation to perform cost estimation and optimization. Estimation may comprise inferring the cost of a candidate data partitioning plan, and optimization may comprise generating an optimal partitioning plan based on the estimated costs of computation and input/output.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.