Boosting DNN Efficiency: Replacing FC Layers with Graph Embeddings for Hardware Acceleration

Fully connected (FC) layers present a significant bottleneck in the throughput of Convolutional Neural Networks (CNNs) and Transformers when deployed on hardware accelerators such as FPGAs, ASICs, and analog AI chips. This paper proposes a novel approach to mitigate this limitation by replacing FC layers with sparse graph embedding techniques derived from Low-Density Parity-Check (LDPC) codes, enabling robust weight compression. Using VGG16 as a case study, we demonstrate that our method reduces the parameters in the FC layers from 123,642,856 to just 64,000-a 1,931-fold reduction-while maintaining high-precision classification performance. Following aggressive pruning (72% weight reduction), our LDPC-based embeddings, combined with 8-bit fixed-point quantization, achieve an accuracy of 70.21%, compared to 70.89% achieved by unpruned float32. On the Xilinx Alveo U50 platform, this translates to processing up to 1119 images per second with parallel computation of 64-dimensional embeddings. Moreover, by replacing the top FC layers with the proposed graph embedding method and leveraging fast analog feature extraction (via supervised or random projections), our approach demonstrates the potential to achieve peak performance on the Alveo U50 of 4.8 million images per second for float32 100-class classification and 39.86 million images per second for 10-class classification using 16-bit fixed-point representation. These findings underscore the capability of graph embeddings to significantly enhance DNN efficiency on data center and resource-constrained edge AI hardware platforms. We hypothesize that exploiting the inherent sparsity of graph embeddings, combined with the energy efficiency of analog computation for feature extraction, can yield substantial reductions in CNN, Transformer and Diffusion DNN inference power consumption, potentially surpassing GPU efficiency by three to four orders of magnitude. This paper presents preliminary results and outlines directions for future research to validate this hypothesis.