Methods and systems to adapt PID coefficients through reinforcement learning

US12153385B2 · kind B2 · utility

Assignees

• SONY GROUP CORPORATION · JP
• SONY CORPORATION OF AMERICA · US

Inventors

• Samuel Barrett · Everett, US
• James MacGlashan · Luthers Corners, US
• Varun Kompella · Aachen, DE
• Peter R. Wurman · Acton, US
• Goker Erdogan · London, GB
• Fabrizio Santini · Brookline, US

Key dates

• Filing date: May 7, 2021
• Grant date: Nov 26, 2024
• Expiry date: May 19, 2042

Classification

• Technology area (CPC G): Physics
• CPC primary: G06N20/00
• WIPO field: Control
• WIPO sector: Instruments

Abstract

Systems and methods are used to adapt the coefficients of a proportional-integral-derivative (PID) controller through reinforcement learning. The approach for adapting PID coefficients can include an outer loop of reinforcement learning where the PID coefficients are tuned to changes in the environment and an inner loop of PID control for quickly reacting to changing errors. The outer loop can learn and adapt as the environment changes and be configured to only run at a predetermined frequency, after a given number of steps. The outer loop can use summary statistics about the error terms and any other information sensed about the environment to calculate an observation. This observation can be used to evaluate the next action, for example, by feeding it into a neural network representing the policy. The resulting action is the coefficients of the PID controller and the tunable parameters of things such as the filters.