Adaptive map-reduce pipeline with dynamic thread allocations
US9684513B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Sep 21, 2015 |
| Grant date | Jun 20, 2017 |
| Priority date | — |
| Expiry date | Sep 21, 2035 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2212/62
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
In an approach to adaptively pipeline a MapReduce job, a processor receives one or more data records from a storage and inserts the one or more data records into a map queue, wherein a size of the map queue is adaptive to one or more utilizations of one or more resources in the processor. One or more processors apply a map function to the one or more data records in the first buffer and sort the records that are output from the map function and store the sorted records. One or more processors receive and insert the sorted records into a reduce queue, wherein a size of the reduce queue is adaptive to one or more utilizations of resources in the one or more processors. One or more processors apply a reduce function to the sorted records in the reduce queue and store a result in a storage.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.