Machine learning deployment platform
US12412109B2 · kind B2 · utility
0Cited by
0References
19Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Feb 3, 2022 |
| Grant date | Sep 9, 2025 |
| Priority date | — |
| Expiry date | May 2, 2044 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N20/00
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An inference server is capable of receiving a plurality of inference requests from one or more client systems. Each inference request specifies one of a plurality of different endpoints. The inference server can generate a plurality of batches each including one or more of the plurality of inference requests directed to a same endpoint. The inference server also can process the plurality of batches using a plurality of workers executing in an execution layer therein. Each batch is processed by a worker of the plurality of workers indicated by the endpoint of the batch.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.