Hardware-Aware Network Adaptation using Width and Depth Shrinking including Convolutional and Fully Connected Layer Merging

Deploying complex Convolutional Neural Networks (CNNs) on edge devices with limited resources and battery power presents challenges due to their high computational and memory demands. Many existing algorithmic compression techniques for these CNNs struggle to work effectively on hardware as they don’t consider the hardware’s information. This results in inefficient memory usage and makes real-time execution difficult. We observed that we can shrink the model’s size by modifying its width and depth to make it adaptable to edge devices. This helps in reducing the necessity to store interim data off-chip. We demonstrated the effectiveness of our proposed methodology by applying “Width Shrinking” and “Depth Shrinking” (ConvMerge and FcMerge) methodologies to AlexNet and VGG16 networks. We then compared the results with those of trained models on different hardware platforms inluding Nvidia AGX-Xavier (GPU) and AMD-Xilinx ZYNQ Ultrascale ZCU102 System-on-Chip (SoC) (A53 CPU and Programmable Logic (PL)). Our findings indicate that our proposed methodology can significantly reduce the model size by upto $98 \%$ and decrease the inference time by $30 \%$ and $67 \%$ on edge GPU and CPU platforms respectively, with insignificant change in accuracy for AlexNet. Similarly, the VGG16 shows model size reduction by $98 \%$, inference time reduction on CPU by $77 \%$ and on GPU by $87 \%$. On FPGA-SoC, we found that our proposed methodology cuts-down on off-chip feature map data transfer, slashing the total number of transactions by $98 \%$.