Optimizations for machine learning data processing pipeline

US11797885B2 · kind B2 · utility

Assignee

• SAP SE · DE

Inventors

• Steven Jaeger · Heidelberg, DE
• Isil Pekel · Heidelberg, DE
• Manuel Zeise · Karlsruhe, DE

Key dates

• Filing date: Sep 24, 2020
• Grant date: Oct 24, 2023
• Expiry date: Jun 6, 2041

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N3/082
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

A data processing pipeline may be generated to include an orchestrator node, a preparator node, and an executor node. The preparator node may generate a training dataset. The executor node may execute machine learning trials by applying, to the training dataset, a machine learning model and/or a different set of trial parameters. The orchestrator node may identify, based on a result of the machine learning trials, a machine learning model for performing a task. The execution of the data processing pipeline may be optimized. Examples of optimizations include pooling multiple machine learning trials for execution at a single executor node, executing at least some machine learning trials using a sub-sample of the training dataset, and adjusting a proportion of trial parameters sampled from a uniform distribution to avoid a premature convergence to a local minima within the hyper-parameter space for generating the machine learning model.