Systems and/or methods for leveraging in-memory storage in connection with the shuffle phase of MapReduce
US10990288B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 1, 2014 |
| Grant date | Apr 27, 2021 |
| Priority date | — |
| Expiry date | Feb 23, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F3/0673
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Certain example embodiments relate to a computer system for performing a map reduce sequence. Nodes therein include at least one processor and memory and are divided into at least mapper and reducer nodes. Each mapper node executes a map function on input to generate intermediate output elements. Each said intermediate output element includes a first key-value pair. Each element key includes associated map and reduce task identifiers. Each element value includes substantive data, organized as another key-value pair. The intermediate output elements are stored to memory. Each reducer node: retrieves at least intermediate output element values from the memory of a given mapper node, using specified map and reduce task identifiers; stores the retrieved element values to its memory; executes a reduce function on the retrieved element values, in order; and outputs a result from the reduce function for the map reduce sequence. Disk operations advantageously are reduced or eliminated.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.