Platform-Aware Accelerated Compilation for Edge Via Fully Connected and Convolutional Layer Merging

Platform-aware optimization has revolutionized the Convolutional Neural Networks (CNNs) deployment on edge platforms. However, current strategies primarily focus on reducing Multiply-Accumulate (MAC) operations at the expense of in-creased compilation times. Extracting hardware-specific metrics like latency, from deep-layered models remains a time-intensive process, limiting rapid deployment. To address this, we propose “Fully Connected Layer Merging” (FCLM) and “Convolutional Layer Merging” (CLM) methodologies, which independently consolidate fully connected and convolutional layers in CNNs, enhancing network efficiency with an insignificant change in accuracy. Our work is evaluated on AlexNet and VGG16 models trained on CIFAR-10 and Fruit-360 datasets, respectively. Our proposed methodologies showed a significant reduction in compile time, model size and MACs. To exploit the benefits of platform-awareness, we integrated our work with NetAdapt. We achieved 94% and 97% reduction in compile time for AlexNet and VGG16, respectively. Along with this, we attained substantial reductions in model size and MACs up to 97% and 60% for AlexNet, 97% and 95% for VGG16, respectively. Furthermore, inference on Nvidia AGX Xavier GPU showed a 33% reduction in time for AlexNet and 77% for VGG16. Hardware validation on AMD-Xilinx ZYNQ Ultrascale ZCU102 board shows a 99% reduction in Processing System to Programmable Logic (PS- PL) transactions, effectively minimizing power consumption during CNN inference.