Parallel file system with metadata distributed across partitioned key-value store
US9767107B1 · kind B1 · utility
Assignees
Inventors
Key dates
| Filing date | Jun 29, 2013 |
| Grant date | Sep 19, 2017 |
| Priority date | — |
| Expiry date | Apr 21, 2034 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/1858
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.