Proxy servers for managing queries to large language models
US12261827B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 19, 2024 |
| Grant date | Mar 25, 2025 |
| Priority date | — |
| Expiry date | Jan 19, 2044 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L41/16
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Systems, methods, and apparatus, including computer programs encoded on a computer storage medium for managing network traffic to and from a server configured to: (i) receive, from a client device, a query in a natural language, and (ii) generate a response to the query in the natural language. In one aspect, a method includes: receiving, from the client device via a network connection, a network message including a new query for the server; processing the new query, using a text encoder, to generate an embedding vector of the new query; identifying, from amongst multiple entries of a vector database, a particular entry based on a similarity metric between: (i) the embedding vector of the new query, and (ii) an embedding vector of a particular query stored in the particular entry; and determining whether the similarity metric is greater than a threshold similarity value.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.