Upscale Layer Acceleration on Existing AI Hardware

Upscaling layers are important components of modern deep learning networks but often pose computational challenges for hardware accelerators. This paper addresses this issue by introducing a novel layer-replacement technique to efficiently process upscaling layers using existing hardware-supported operations like depthwise convolutions and maximum pooling. To minimize the number of replacement layers, we propose an efficient layer number reduction algorithm. Experimental results on 4 deep neural networks demonstrate a significant speedup ranging from 1.58x to 32.88x compared to the original HW/SW execution approach, and from 3.65x to 19.21x compared to the softwareonly solution, with minimal hardware overhead (0.068% more FPGA LUTs). Notably, our technique introduces no numerical errors and maintains comparable input data processing quality to the original network.