Layered measurement, grading and evaluation of pretrained artificial intelligence models
US12135949B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 28, 2024 |
| Grant date | Nov 5, 2024 |
| Priority date | — |
| Expiry date | Jun 28, 2044 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/106
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems and methods for evaluating a pre-trained artificial intelligence (AI) model using layered prompts. The system obtains a set of application domains in which the AI model will be used, and a set of guidelines that define one or more operational boundaries of the AI model. The system determines a set of layers, where each layer is associated with corresponding guidelines and mapped to a set of variables and benchmarks. Each variable represents an attribute within the guidelines and each benchmark indicates the degree of satisfaction of the AI model with the guidelines. The AI model is dynamically evaluated against these benchmarks using a series of assessments. Subsequent assessments are dynamically constructed based on the outcomes of previous assessments. Scores are assigned to the AI model for each layer by comparing the expected and actual responses. The results are then displayed in a graphical user interface (GUI).
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.