Analog-AI Hardware Accelerators for Low-Latency Transformer-Based Language Models (Invited)
Analog Non-Volatile Memory-based accelerators offer highthroughput and energy-efficient Multiply-Accumulate operations for the large Fully-Connected layers that dominate Transformer-based Large Language Models (LLMs). We describe recent chip-demo and architectural efforts, quantify the unique benefits of Fully- (rather than Partially-) Weight-Stationary systems, and discuss factors affecting latency of token-processing and generation.
Authors
- Kohji Hosokawa (ORCID: https://orcid.org/0009-0009-8086-8144)
- Shilpa Jain (ORCID: https://orcid.org/0000-0002-6033-6062)
- Geoffrey W. Burr (ORCID: https://orcid.org/0000-0001-5717-2549)
- Charles Mackin (ORCID: https://orcid.org/0000-0001-8413-5583)
- Atsuya Okazaki (ORCID: https://orcid.org/0000-0002-5275-5224)
- Abu Sebastian (ORCID: https://orcid.org/0000-0001-5603-5243)
- Hadjer Benmeziane (ORCID: https://orcid.org/0000-0002-5259-0749)
- Andrea Fasoli (ORCID: https://orcid.org/0000-0001-6892-5139)
- Jose Luquin (ORCID: https://orcid.org/0009-0005-9539-1386)
- Manuel Le Gallo (ORCID: https://orcid.org/0000-0003-1600-6151)
- Kaoutar El Maghraoui (ORCID: https://orcid.org/0000-0002-1967-8749)
- Yasuteru Kohda
- Julian Büchel (ORCID: https://orcid.org/0000-0001-9495-7150)
- Vijay Narayanan (ORCID: https://orcid.org/0009-0008-8433-963X)
- Pritish Narayanan (ORCID: https://orcid.org/0000-0002-3176-0059)
- Masatoshi Ishii (ORCID: https://orcid.org/0000-0003-0794-7232)
- Corey Lammie (ORCID: https://orcid.org/0000-0001-5564-1356)
- Elena Ferro
- Alexander Friz
- Irem Boybat (ORCID: https://orcid.org/0000-0002-4255-8622)
- Hsinyu Tsai (ORCID: https://orcid.org/0000-0002-3971-097X)
- William A. Simon (ORCID: https://orcid.org/0000-0002-1832-4690)
- Athanasios Vasilopoulos
- A. Chen (ORCID: https://orcid.org/0009-0005-3814-7264)
- Timothy Philicelli
Institutions
- IBM Research - Almaden (US)
- IBM Research - Zurich (CH)
- IBM Research - Tokyo (JP)
Publication Details
- Published
- 2025-04-13
- DOI
- https://doi.org/10.1109/cicc63670.2025.10983594
- Citations
- 1
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 2.48
Funders
- Intercollegiate Center for Management Science