Atomic incremental load for map-reduce systems on append-only file systems
US9424271B2 · kind B2 · utility
Assignee
Inventor
Key dates
| Filing date | Aug 30, 2012 |
| Grant date | Aug 23, 2016 |
| Priority date | — |
| Expiry date | Jun 21, 2033 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/1873
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Augmenting data files in a repository of an append-only file system comprises maintaining metadata corresponding to each data file for tracking a logical end-of-file (EOF) for each data file for appending. A global versioning mechanism for the metadata allows selecting the current version of the metadata to read for performing an append job for a set of data files. Each append job comprises multiple append tasks. For each successful append job, the global versioning mechanism increments a valid metadata version to use for each data file appended. Said valid metadata version indicates the logical EOF corresponding to a new physical EOF for each of the data files appended.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.