Server system with AI accelerator apparatuses using in-memory compute chiplet devices for transformer workloads
US12353985B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 13, 2023 |
| Grant date | Jul 8, 2025 |
| Priority date | — |
| Expiry date | Jan 17, 2044 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/0455
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A server system with AI accelerator apparatuses using in-memory compute chiplet devices. The system includes a plurality of multiprocessors each having at least a first server central processing unit (CPU) and a second server CPU, both of which are coupled to a plurality of switch devices. Each switch device is coupled to a plurality of AI accelerator apparatuses. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a CPU, and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.