Framework for testing machine learning workflows

US11392469B2 · kind B2 · utility

Assignee

• MICROSOFT TECHNOLOGY LICENSING, LLC · US

Inventors

• Ali Mohamed · San Jose, US
• Manish Swaminathan · Fremont, US
• Shunlin Liang · San Jose, US
• Prateek Sachdev · Sunnyvale, US
• Vivek Desai · Sunnyvale, US
• Adam Robert Peck · Redwood City, US
• Sunny Sanjiv Ketkar · Fremont, US

Key dates

• Filing date: Jun 20, 2019
• Grant date: Jul 19, 2022
• Expiry date: Sep 24, 2040

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N7/01
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

The disclosed embodiments provide a system for testing machine learning workflows. During operation, the system obtains a configuration for a staging test of a machine learning model, wherein the configuration includes a model name for the machine learning model, a duration of the staging test, and a use case associated with the machine learning model. Next, the system selects a staging test host for the staging test. The system then deploys the staging test on the staging test host in a staging environment, wherein the deployed staging test executes the machine learning model based on live traffic received from a production environment. After the staging test has completed, the system outputs a set of metrics representing a system impact of the machine learning model on the staging test host.