Counterfactual policy evaluation of model performance
US12307338B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Dec 21, 2023 |
| Grant date | May 20, 2025 |
| Priority date | — |
| Expiry date | Dec 21, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06Q30/01
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An online system uses an offline iterative clustering process to evaluate the performance of a set of content selection frameworks. To perform an iteration of the iterative clustering process, an online system clusters the testing example data into a set of clusters. An online system computes a set of framework scores for each of the generated clusters. An online system computes an improvement score for each cluster based on the performance scores of the clusters. To determine whether to perform another iteration, an online system computes an aggregated improvement score based on the improvement scores of the clusters. If an online system determines that the aggregated improvement score does not meet the threshold, an online system performs another iteration of the process above. When an online system finishes the iterative process, an online system outputs the improvement scores of the most-recent iteration.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.