System and method for robust evaluation of the user experience in automated spoken dialog systems
US8520808B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 8, 2009 |
| Grant date | Aug 27, 2013 |
| Priority date | — |
| Expiry date | Jun 27, 2032 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG10L15/01
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A single, subjective numerical rating to evaluate the performance of a telephone-based spoken dialog system. This CE rating is provided by expert human listeners who have knowledge of the design of the dialog system. Different human raters can be trained to achieve a satisfactory level of agreement. Furthermore, a classifier trained on ratings by human experts can reproduce the human ratings with the same degree of consistency. More calls can be given a CE rating than would be possible with limited human resources. More information can be provided about individual calls, e.g., to help decide between two disparate ratings by different human experts.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.