Techniques for fast IO and low memory consumption while using erasure codes
US10715184B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 11, 2017 |
| Grant date | Jul 14, 2020 |
| Priority date | — |
| Expiry date | Oct 1, 2038 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH03M13/152
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods and systems for improving the read and write performance of a distributed file system while limiting memory usage are described. The type of error correcting scheme applied to data, the partitioning of the data into data chunks, and the sizes of data slices within each of the data chunks used for storing electronic files within the distributed file system may be dynamically adjusted over time to optimize for fast IO performance while limiting memory usage (e.g., requiring less than 256 MB of RAM to generate and store code blocks). The file size of an electronic file to be stored, the amount of available memory for generating code blocks, and the amount of available disk space to store the electronic file may be used to set the data sizes of the data slices and the type of erasure code applied to data blocks associated with the data slices.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.