Distributed batch processing of non-uniform data objects
US11010103B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 20, 2019 |
| Grant date | May 18, 2021 |
| Priority date | — |
| Expiry date | Sep 12, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F3/0644
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
The described methods, systems, and other aspects can advantageously provide balanced multi-stage processing of non-uniform object data. An example method may receive a list of buckets. Each of the buckets in the list of buckets can store one or more restorable objects. The method further comprises distributing the list of buckets to the two or more second nodes; determining a number of the one or more restorable objects in each bucket; determining a size of the one or more restorable objects in each bucket; generating batches of to-be-restored data objects based on the determined number of the one or more restorable objects in each bucket and the determined size of the one or more restorable objects in each bucket; and distributing the batches among the two or more second nodes for storage-related task processing.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.