System, method, and computer-readable medium for dynamic detection and management of data skew in parallel join operations
US7941424B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 30, 2008 |
| Grant date | May 10, 2011 |
| Priority date | — |
| Expiry date | Jul 16, 2029 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/2456
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A system, method, and computer-readable medium for dynamic detection and management of data skew in parallel join operations are provided. Receipt of an excessive number of redistributed rows by a processing module is detected thereby identifying the processing module as a hot processing module. Other processing modules then terminate redistribution of rows to the hot processing module and maintain rows of a skewed table of the join operation that would be redistributed to the hot processing module in a local spool. Rows of a smaller table that would be redistributed to the hot processing module are duplicated to each processing module involved in the join operation. Rows of tables that are to be redistributed by a processing module to any processing module excluding the hot processing module are redistributed accordingly and maintained locally by the processing module. The join operation is completed by merging results of local join data sets of each processing module.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.