Batching inputs to a machine learning model
US10789544B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 5, 2016 |
| Grant date | Sep 29, 2020 |
| Priority date | — |
| Expiry date | Mar 21, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2209/548
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for batching inputs to machine learning models. One of the methods includes receiving a stream of requests, each request identifying a respective input for processing by a first machine learning model; adding the respective input from each request to a first queue of inputs for processing by the first machine learning model; determining, at a first time, that a count of inputs in the first queue as of the first time equals or exceeds a maximum batch size and, in response: generating a first batched input from the inputs in the queue as of the first time so that a count of inputs in the first batched input equals the maximum batch size, and providing the first batched input for processing by the first machine learning model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.