Configuration map based sharding for containers in a machine learning serving infrastructure
US12073258B2 · kind B2 · utility
1Cited by
1References
18Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | May 28, 2021 |
| Grant date | Aug 27, 2024 |
| Priority date | — |
| Expiry date | Oct 15, 2042 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L67/1004
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A machine learning serving infrastructure implementing a method of receiving or detecting an update of container metrics including resource usage and serviced requests per model or per container, processing the container metrics per model or per container to determine recent resource usage and serviced requests per model or per container, and rebalancing distribution of models to a plurality of containers to decrease a detected load imbalance between containers or a stressed container in the plurality of containers.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.