Extensible data transformation authoring and validation system

US10261763B2 · kind B2 · utility

Assignee

• Palantir Technologies Inc. · US

Inventors

• Robert Fink · Huntsville, US
• Matthew Cheah · Mountain View, US
• Mingyu Kim · Anyang-si, KR
• Lynn Cuthriell · San Francisco, US
• Divyanshu Arora · New York, US
• Justin Uang · New York, US
• Jared Newman · Boise, US
• Jakob Juelich · Brandenburg, DE
• Kevin Chen · Stanford, US
• Mark Elliot · Kingston, US
• Michael Nazario · Menlo Park, US

Key dates

• Filing date: Dec 12, 2017
• Grant date: Apr 16, 2019
• Expiry date: Dec 12, 2037

Classification

• Technology area (CPC G): Physics
• CPC primary: G06F21/30
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Data transformation in a distributed system of applications and data repositories is described. The subsystems for the overall framework are distributed, thereby allowing for customization to require only isolated changes to one or more subsystems. In one embodiment, a source code repository is used to receive and store source code. A build subsystem can retrieve source code from the source code repository and build it, using one or more criteria. By building the source code, the build subsystem can generate an artifact, which is executable code, such as a JAR or SQL file. Likewise, by building the source code, the build subsystem can generate one or more job specifications for executing the executable code. In one embodiment, the artifact and job specification may be used to launch an application server in a cluster. The application server can then receive data transformation instructions and execute the data transformation instructions.