Performance of neural networks under distribution shift
US12172670B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 15, 2022 |
| Grant date | Dec 24, 2024 |
| Priority date | — |
| Expiry date | Jun 23, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06V2201/06
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods and systems of estimating an accuracy of a neural network on out-of-distribution data. In-distribution accuracies of a plurality of machine learning models trained with in-distribution data are determined. The plurality of machine learning models includes a first model, and a remainder of models. In-distribution agreement is determined between (i) an output of the first machine learning model executed with an in-distribution dataset and (ii) outputs of a remainder of the plurality of machine learning models executed with the in-distribution dataset. The machine learning models are also executed with an unlabeled out-of-distribution dataset, and an out-of-distribution agreement is determined. The in-distribution agreement is compared with the out-of-distribution agreement. Based on a result of the comparison being within a threshold, an accuracy of the first machine learning model on the unlabeled out-of-distribution dataset is estimated based on (i) the in-distribution accuracies, (ii) the in-distribution agreement, and (iii) the out-of-distribution agreement.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.