Dynamic composition of data pipeline in accelerator-as-a-service computing environment
US10776164B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 30, 2018 |
| Grant date | Sep 15, 2020 |
| Priority date | — |
| Expiry date | Jan 17, 2039 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/044
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are provided to decouple data pipeline tasks from an execution flow of a high-performance computing task (e.g., distributed deep model training) in a distributed computing system. For example, a method includes receiving a client request to provision resources for executing a computing job, provisioning accelerator resources of one or more accelerator server nodes in the distributed computing system to perform tasks associated with an execution flow of the computing job, and provisioning a logical nodes within the distributed computing system to compose a data flow pipeline which is configured to perform data flow operations associated with the computing job for providing data to the provisioned accelerator resources to perform the tasks associated with the execution flow of the computing job. The data flow operations include, e.g., data storage input/output operations, data pre-processing operations, and data staging operations, which are decoupled from the execution flow of the computing job.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.