Non-partitioned field query optimization in data lakes
US12373429B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 20, 2023 |
| Grant date | Jul 29, 2025 |
| Priority date | — |
| Expiry date | Nov 20, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/278
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A mapping between values of a non-partitioning field and containing Parquet files of a data lake is generated. The containing Parquet files include the values of the non-partitioning field. The data lake is organized into partitions and Parquet files. Each partition includes a subset of the Parquet files. A query of the data lake that includes the non-partitioning field in a filter is received. A subset of the containing Parquet files corresponding to the filter is identified based on the mapping. The query is rewritten into a rewritten query that includes filters that are based on the subset of the containing Parquet files. Results of executing the rewritten query are transmitted or displayed.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.