TP-DCIM: Transposable Digital SRAM CIM Architecture for Energy-Efficient and High Throughput Transformer Acceleration

To accelerate the execution of transformer models, compute-in-memory (CIM) has been widely adopted. However, the CIM architecture has the drawback of fixed one-way computing structure supporting only horizontal input sharing vertical accumulation (HIVA). So, it faces two major obstacles: 1) Matrix transposition processing needs large hardware overheads, 2) A fixed dataflow incurs CIM underutilization problem, degrading throughput. In this paper, we present a digital SRAM CIM (DCIM)-based transformer accelerator that supports two-way computing: HIVA and vertical input sharing horizontal accumulation (VIHA). The proposed two-way computing DCIM macro features a novel 9T SRAM bitcell and transposable adder tree, which efficiently reduces matrix transposition costs. We also present a novel dataflow to improve overall self-attention latency by resolving CIM underutilization and hiding dynamic input generation latency. In addition, CIM-friendly computation skipping scheme is exploited to enhance energy efficiency with negligible accuracy loss. The simulation results show that the proposed DCIM-based accelerator achieves up to 2.5x latency improvement and 24% energy savings compared to previous CIM-based accelerators.