TT-RRAM: Joint Improvement of Sparsity and Repetition in Tensor-Train Inference With Fused Processing on RRAM Accelerators

Deep neural networks compressed using Tensor-Train decomposition , known as TT-DNNs, significantly reduce model size to decrease storage requirements but suffer from more frequent data movement during inference. To address this issue, RRAM-based DNN accelerators, by leveraging computing-in-memory, can mitigate the data movement and efficiently execute the VMM operations required during inference, making them an ideal candidate for accelerating the inference for TT-DNNs. However, RRAM-based accelerators face three challenges in TT-format DNN inference: (1) weights distribution on the RRAM crossbar with low sparsity and repetition; (2) low input sparsity; and (3) high data dependencies and storage overhead during inference. To address these challenges, we propose three corresponding methods: (1) Base-Offset Splitting (BOS) to improve the weights distribution on the crossbar, thereby enhancing performance potential; (2) Inverse Input Reusing (IIR) to enhance the sparsity of inputs during inference; and (3) Fused Scheduling (FS) to reorganize the computation order and leveraging the parallel processing capabilities of the RRAM-based accelerator to improve overall inference efficiency while also reducing data dependencies and storage overhead. Experimental results demonstrate that our proposed methods achieve a 3.44× to 5.53× performance improvement and 55.6% to 67.1% energy savings compared to the state-of-the-art RRAM-based accelerator, while also reducing storage overhead by 66.7% to 99.1%.