AI accelerator apparatus using in-memory compute chiplet devices for transformer workloads
US12271321B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 24, 2023 |
| Grant date | Apr 8, 2025 |
| Priority date | — |
| Expiry date | Oct 24, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2213/0026
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.