System for routing machine learning model inferences
US11170309B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Nov 22, 2017 |
| Grant date | Nov 9, 2021 |
| Priority date | — |
| Expiry date | Sep 10, 2040 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L43/50
- WIPO fieldDigital communication
- WIPO sectorElectrical engineering
Abstract
A machine learning model inference routing system in a machine learning service is described herein. The machine learning model inference routing system includes load balancer(s), network traffic router(s), an endpoint registry, and a feedback processing system that collectively allow the machine learning model inference routing system to adjust the routing of inferences based on machine learning model accuracy, demand, and/or the like. In addition, the arrangement of components in the machine learning model inference routing system enables the machine learning service to perform shadow testing, support ensemble machine learning models, and/or improve existing machine learning models using feedback data.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.