Using index partitioning and reconciliation for data deduplication
US9785666B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jul 13, 2015 |
| Grant date | Oct 10, 2017 |
| Priority date | — |
| Expiry date | Jul 13, 2035 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/24556
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
The subject disclosure is directed towards a data deduplication technology in which a hash index service's index is partitioned into subspace indexes, with less than the entire hash index service's index cached to save memory. The subspace index is accessed to determine whether a data chunk already exists or needs to be indexed and stored. The index may be divided into subspaces based on criteria associated with the data to index, such as file type, data type, time of last usage, and so on. Also described is subspace reconciliation, in which duplicate entries in subspaces are detected so as to remove entries and chunks from the deduplication system. Subspace reconciliation may be performed at off-peak time, when more system resources are available, and may be interrupted if resources are needed. Subspaces to reconcile may be based on similarity, including via similarity of signatures that each compactly represents the subspace's hashes.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.