Cloud architecture for reinforcement learning

US12001320B2 · kind B2 · utility

Assignee

• MICROSOFT TECHNOLOGY LICENSING, LLC · US

Inventors

• Xiaoyan Liu · Beijing, CN
• Steve K. Lim · Redmond, US
• Taylor Paul Spangler · Kirkland, US
• Kashyap Maheshkumar Patel · Bellevue, US
• Marc Mas Mezquita · Redmond, US
• Levent Ozgur · Seattle, US
• Timothy James Chapman · Bellevue, US

Key dates

• Filing date: Jun 30, 2022
• Grant date: Jun 4, 2024
• Expiry date: Jun 30, 2042

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N7/01
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

The technology described herein provides a cloud reinforcement-learning architecture that allows a single reinforcement-learning model to interact with multiple live software environments. The live software environments and the single reinforcement-learning model run in a distributed computing environment (e.g., cloud environment). The single reinforcement-learning model may run on a first computing device(s) with a graphical processing unit (GPU) to aid in training the single reinforcement-learning model. At a high level, the single reinforcement-learning model may receive state telemetry data from the multiple live environments. The single reinforcement-learning model selects an available action for each set of state telemetry data received and communicates the selection to appropriate the test agent. The test agent then facilitates completion of the action within the software instance being tested in the live environment. A reward is then determined for the action. The single reinforcement-learning model may then be trained.