Distributed resource-aware training of machine learning pipelines
US11829799B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 13, 2020 |
| Grant date | Nov 28, 2023 |
| Priority date | — |
| Expiry date | Nov 28, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2209/5019
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method, a structure, and a computer system for predicting pipeline training requirements. The exemplary embodiments may include receiving one or more worker node features from one or more worker nodes, extracting one or more pipeline features from one or more pipelines to be trained, and extracting one or more dataset features from one or more datasets used to train the one or more pipelines. The exemplary embodiments may further include predicting an amount of one or more resources required for each of the one or more worker nodes to train the one or more pipelines using the one or more datasets based on one or more models that correlate the one or more worker node features, one or more pipeline features, and one or more dataset features with the one or more resources. Lastly, the exemplary embodiments may include identifying a worker node requiring a least amount of the one or more resources of the one or more worker nodes for training the one or more pipelines.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.