Fast and scalable multi-tenant serve pool for chatbots
US12169763B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 13, 2021 |
| Grant date | Dec 17, 2024 |
| Priority date | — |
| Expiry date | Jul 2, 2043 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L51/18
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques are disclosed for providing a scalable multi-tenant serve pool for chatbot systems. A query serving system (QSS) receives a request to serve a query for a skillbot. The QSS includes: (i) a plurality of deployments in a serving pool, and (ii) a plurality of deployments in a free pool. The QSS determines whether a first deployment from the plurality of deployments in the serving pool can serve the query based on an identifier of the skillbot. In response to determining that the first deployment cannot serve the query, the QSS selects a second deployment from the plurality of deployments in the free pool to be assigned to the skillbot, and loads a machine-learning model associated with the skillbot into the second deployment, wherein the machine-learning model is trained to serve the query for the skillbot. The query is served using the machine-learning model loaded into the second deployment.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.