Export data from tables into partitioned folders on an external data lake
US11347775B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Aug 31, 2021 |
| Grant date | May 31, 2022 |
| Priority date | — |
| Expiry date | Aug 31, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/254
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.