DPIM: A 19.36 TOPS/W 2T1C eDRAM Transformer-in-Memory Chip with Sparsity-Aware Quantization and Heterogeneous Dense-Sparse Core

This paper presents DPIM, the first 2T1C eDRAM Transformer-in-memory chip. Its high-density eDRAM cell supports large-capacity processing-in-memory (PIM) macros of 1.38 $\mathrm{Mb} / \mathrm{mm}^{2}$, reducing external memory access. DPIM adopts a sparse-aware quantization scheme to entire layers of Transformer, which quantizes the model to 8-bit integer (INT8) with a minimal accuracy drop of $2 \%$ in the BERT-large model on the GLUE dataset while increasing the bit-slice sparsity ratio of both weight and activation from dense matrices to $83.3 \%$ and $88.4 \%$, respectively. Its heterogeneous PIM macro supports intensive dense matrix multiplications with an extreme to moderate range of sparse matrix multiplications with a peak throughput of 3.0312.12 TOPS, enhancing the efficiency up to 4.84-19.36 TOPS/W.

Authors

Institutions

Publication Details

Published
2024-09-09
DOI
https://doi.org/10.1109/esserc62670.2024.10719539
Citations
5
Primary Topic
Advanced Memory and Neural Computing
Type
article
Field-Weighted Citation Impact
1.04
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

DPIM: A 19.36 TOPS/W 2T1C eDRAM Transformer-in-Memory Chip with Sparsity-Aware Quantization and Heterogeneous Dense-Sparse Core

Gunmo Koo, Suk Jin Lee, Hoichang Jeong, Kunhee Cho et al.
5 citations
Advanced Memory and Neural Computing
1.04
article

DPIM: A 19.36 TOPS/W 2T1C eDRAM Transformer-in-Memory Chip with Sparsity-Aware Quantization and Heterogeneous Dense-Sparse Core

Gunmo Koo, Suk Jin Lee, Hoichang Jeong, Kunhee Cho, Jaemin Kim, Joo‐Hyung Chae, Ki-Soo Lee, Keonhee Park, Hyunjun Cho, Jae‐Jin Kim, Sungwoong Yune, Jaeyoung Kim, Jooyoung Kim, Donghyuk Kim, Jong‐Chan Lee, Yuli Han, Kyuho Lee, Chanheum Han, Seung-Jae Yoo
article en
5 citations

Abstract

This paper presents DPIM, the first 2T1C eDRAM Transformer-in-memory chip. Its high-density eDRAM cell supports large-capacity processing-in-memory (PIM) macros of 1.38 $\mathrm{Mb} / \mathrm{mm}^{2}$, reducing external memory access. DPIM adopts a sparse-aware quantization scheme to entire layers of Transformer, which quantizes the model to 8-bit integer (INT8) with a minimal accuracy drop of $2 \%$ in the BERT-large model on the GLUE dataset while increasing the bit-slice sparsity ratio of both weight and activation from dense matrices to $83.3 \%$ and $88.4 \%$, respectively. Its heterogeneous PIM macro supports intensive dense matrix multiplications with an extreme to moderate range of sparse matrix multiplications with a peak throughput of 3.0312.12 TOPS, enhancing the efficiency up to 4.84-19.36 TOPS/W.

Korea Advanced Institute of Science and Technology (KR), Kwangwoon University (KR), Kyungpook National University (KR), Ulsan National Institute of Science and Technology (KR)
Affordable and clean energy
Openalex Percentile: Top 23%
Advanced Memory and Neural Computing
1.04
DOISourceOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.