Low latency ingestion into a data system
US11487788B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 18, 2022 |
| Grant date | Nov 1, 2022 |
| Priority date | — |
| Expiry date | Jan 18, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/258
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Described herein are techniques for improving transfer of metadata from a metadata database to a database stored in a data system, such as a data warehouse. The metadata may be written into the metadata database with a version stamp, which is monotonic increasing register value, and a partition identifier, which can be generated using attribute values of the metadata. A plurality of readers can scan the metadata database based on version stamp and partition identifier values to export the metadata to a cloud storage location. From the cloud storage location, the exported data can be auto ingested into the database, which includes a journal and snapshot table.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.