Optimizing Deep Learning Based Autonomous Driving Applications on Edge Devices

With recent advances in computing and sensing technologies, autonomous driving has gained increasing interest and become a promising platform to support the next generation intelligent transportation systems. A critical requirement for autonomous driving systems is to be able to utilize AI and machine learning techniques to make reliable decisions on edge devices in a timely manner. Deploying reliable machine learning models on edge devices in a real-time environment is a challenging task. Real-time applications such as traffic surveillance or traffic sign detection require consistently low latency in order for the device to keep up with its environment. Edge devices are able to compute machine learning tasks without needing to offload computation to a cloud server; however, they often have limited resources, which present challenges for computationally intensive deep learning applications. Therefore, the optimization of neural network models for autonomous driving applications is pivotal for real-time performance on resource-constrained edge devices. In this article, we present a comprehensive study on utilizing deep learning optimization techniques to enable efficient and effective decision-making for autonomous driving applications. Our contributions include the implementation of channel and fine-grained pruning on YOLOv8, direct optimization of detection layers, and the integration of INT8 quantization using NVIDIA TensorRT. These methods significantly improve computational efficiency while preserving the model accuracy. Experimental evaluations on the Jetson Orin Nano demonstrate significant improvements in inference speed and memory utilization with minimal accuracy degradation. This work highlights the feasibility of deploying state-of-the-art object detection models in resource-constrained autonomous driving systems.