Method for shard assignment in a large-scale data processing job
US9298760B1 · kind B1 · utility
216Cited by
3References
21Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Aug 3, 2012 |
| Grant date | Mar 29, 2016 |
| Priority date | — |
| Expiry date | Jun 2, 2034 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/2228
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method for shard assignment in a large-scale data processing job is provided. Datasets are divided into a plurality of shards and the shards are indexed and aggregated into one or more groups. A worker process is initially assigned an indexed shard from a group. The initial assignment can assigned based on a simple algorithm. The worker's subsequent shard assignment is based on the index of the initially assigned shard.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.