Interpretability of deep reinforcement learning models in assistant systems

US11657333B1 · kind B1 · utility

Assignee

• Meta Platforms Technologies, LLC · US

Inventors

• Honglei Liu · San Mateo, US
• Pararth Paresh Shah · Sunnyvale, US
• Wenxuan Li · Mountain View, US
• Wenhai Yang · Menlo Park, US
• Anuj Kumar · Lucknow, IN

Key dates

• Filing date: Apr 19, 2019
• Grant date: May 23, 2023
• Expiry date: Sep 7, 2041

Classification

• Technology area (CPC —): General

Abstract

In one embodiment, a method includes training a target machine-learning model iteratively by accessing training data of content objects, training an intermediate machine-learning model that outputs contextual evaluation measurements based on the training data, generating state-indications associated with the training data, wherein the state-indications comprise user-intents, system actions, and user actions, training the target machine-learning model based on the contextual evaluation measurements, the state-indications, and an action set comprising possible system actions, extracting rules based on the target machine-learning model by a sequential pattern-mining model, generating synthetic training data based on the rules, updating the training data by adding the synthetic training data to the training data, determining if a completion condition is reached for the training, and if the completion condition is reached returning the target machine-learning model, else repeating the iterative training of the target machine-learning model.