Reinforcement Learning Advances in Autonomous Driving: A Detailed Examination of DQN and PPO

One of the most important subdivisions of artificial intelligence, self-driving is widely regarded as the technology that will finally change transportation for the better. This paper involves a review of the current developments in the use of reinforcement learning (RL) to enhance the autonomy of cars; this paper will consider two algorithms, the deep Q-network (DQN) and the Proximal Policy Optimization (PPO). Both the algorithms are considerably powerful in the field of RL; wherein DQN is better suited for the problems with discrete action space, and PPO for continuous control issues. The experiments were performed in the CARLA simulator focused on driving scenes on highways, urban roads, and rural roads, and in various weather conditions and traffic intensities. The findings also show the normality of the performances across both the training and testing sets, and to validate the hypothesis that PPO performs better than DQN, important metrics are presented including the average rewards, collisions, lanes, success rate, and episode’s time steps. PPO policy optimized base technique will achieve more steady and effective learning and that’s why PPO has a high suitability for deeply complex decision-making like self-driving cars. Therefore, our work also contains a comparison with other state-of-the-art autonomous driving techniques to note the excellent position of the PPO method. Furthermore, we discussed the learning patterns and the hyperparameters’ impacts of both DQN and PPO and revealed the advantages and limitations of each. The results confirm the efficiency and flexibility of PPO enhancing its role in real-life RL applications for building future self-driving cars. Consequently, this study enriches the global knowledge on the RL of autonomous driving and provides practical suggestions to improve the learning capabilities of autonomous systems and their safety from further deterioration.