Fast and scalable multi-tenant serve pool for chatbots
US12406203B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 13, 2021 |
| Grant date | Sep 2, 2025 |
| Priority date | — |
| Expiry date | Apr 6, 2044 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L51/18
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are disclosed for providing a scalable multi-tenant serve pool for chatbot systems. A query serving system (QSS) receives a request to serve a query for a new skillbot. The QSS comprises a plurality of deployments, each of which is configured to host a plurality of machine-learning models, each machine-learning model being associated with a skillbot, each deployment including a serving container and a model manager container that hosts a model manager, the serving container including a plurality of sub-containers, each of which hosts one of the machine-learning models downloaded by the model manager. The QSS selects a first deployment to be assigned to the new skillbot based on a first criterion, and loads the machine-learning model associated with the new skillbot into the first deployment. The machine-learning model is trained to serve the query for the new skillbot. The query is served using the machine-learning model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.