Efficient distributed indexing of clumpy data
US10929466B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 15, 2018 |
| Grant date | Feb 23, 2021 |
| Priority date | — |
| Expiry date | Mar 12, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N20/00
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Efficient distributed indexing of clumpy data is disclosed. For example, a plurality of data stores store a plurality of data objects, where a plurality of data references to each of the plurality of data objects is recorded on a distributed index and an index manager executes a processor to temporarily partition a namespace of the plurality of data references into a plurality of temporary partitions. A respective count of data references in each temporary partition is computed. A trend line approximating a distribution of data references is calculated based on the respective counts. A cumulative representation of the distribution of the respective counts is calculated based on the trend line. The namespace is repartitioned based on the cumulative representation resulting in a repartitioned namespace, where the distributed index is divided into index segments based on the repartitioned namespace.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.