System and method for efficient large-scale data processing
US7650331B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 18, 2004 |
| Grant date | Jan 19, 2010 |
| Priority date | — |
| Expiry date | Jan 20, 2026 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F9/544
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A large-scale data processing system and method includes one or more application-independent map modules configured to read input data and to apply at least one application-specific map operation to the input data to produce intermediate data values, wherein the map operation is automatically parallelized across multiple processors in the parallel processing environment. A plurality of intermediate data structures are used to store the intermediate data values. One or more application-independent reduce modules are configured to retrieve the intermediate data values and to apply at least one application-specific reduce operation to the intermediate data values to provide output data.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.