Judgment quality in SBS evaluation
US10387390B2 · kind B2 · utility
0Cited by
0References
20Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Aug 28, 2015 |
| Grant date | Aug 20, 2019 |
| Priority date | — |
| Expiry date | Jan 26, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/335
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Examples of the present disclosure describe systems and methods for using online signals to improve judgment quality in Side-by-Side (SBS) evaluation. In aspects, two or more search result lists may be accessed within a query log. The search result lists may be used to generate and/or determine satisfaction metrics between the search result lists. The satisfaction metrics may be aggregated to automatically generate preference judgments for the search result lists. In some aspects, the preference judgments may be compared to the preference judgments of judges to measure the judgment quality of the judges.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.