Extending a map-reduce framework to improve efficiency of multi-cycle map-reduce jobs
US10102029B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 30, 2015 |
| Grant date | Oct 16, 2018 |
| Priority date | — |
| Expiry date | Dec 21, 2036 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/24553
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods and arrangements for managing map-reduce jobs. There are identified intermediate data produced, in a current map-reduce cycle, by a plurality of nodes in the distributed network, the nodes being selected from the group consisting of: a plurality of map nodes, and a plurality of reducer nodes. There are identified a plurality of classes of data, for classifying the intermediate data. Discrete portions of the intermediate data are classified into respective ones of the classes of data, wherein at least one of the classes of data comprises intermediate data which are processed in a map-reduce cycle subsequent to the current map-reduce cycle. Other variants and embodiments are broadly contemplated herein.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.