Dynamic partitioning of data by occasionally doubling data chunk size for data-parallel applications
US8707320B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Feb 25, 2010 |
| Grant date | Apr 22, 2014 |
| Priority date | — |
| Expiry date | Apr 6, 2032 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F9/505
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Dynamic data partitioning is disclosed for use with a multiple node processing system that consumes items from a data stream of any length and independent of whether the length is undeclared. Dynamic data partitioning takes items from the data stream when a thread is idle and assigns the taken items to an idle thread, and it varies the size of data chunks taken from the stream and assigned to a thread to efficiently distribute work loads among the nodes. In one example, data chunk sizes taken from the beginning of the data stream are relatively smaller than data chunk sizes taken towards the middle or end of the data stream. Dynamic data partitioning employs a growth function where chunks have a size related to single aligned cache lines and efficiently increases the size of the data chunks to occasionally double the amount of data assigned to concurrent threads.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.