Exploring Algorithmic Design Choices for Low Latency CNN Deployment

Convolutional Neural Networks (CNN s) have demonstrated significant success in advancing image and video processing technologies, significantly outperforming traditional methods in both accuracy and efficiency. However, deploying CNN s effectively across diverse hardware platforms often faces the challenge of latency, which can critically impact real-time processing applications. In this work, we explore algorithmic design choices aimed at reducing latency in CNN deployments. We implement five convolution algorithms using SYCL and inte-grate them into three popular CNN models: VGG 16, Resnetl0l, and Inception V 4. By replacing the standard PyTorch Conv2d function with our SY CL- based implementations, we evaluate the execution time of each convolution layer and the overall model on G PU s. Our extensive experiments benchmark the performance of these algorithms against the baseline implementations of the PyTorch and Pytorch Extension for Intel. The results demonstrate significant improvements in execution time, underscoring the potential of these algorithmic choices for achieving low latency in CNN deployments.