Enhancing AWS DeepRacer Performance: A Study on Reward Functions, Action Spaces, and Hyperparameter Tuning

This paper investigates the impact of reward functions, action space configurations, and hyperparameter tuning on AWS DeepRacer models for autonomous driving. Our experiments compare discrete and continuous action spaces, identifying configurations that maximize performance. The findings demonstrate that continuous action spaces, combined with custom reward functions improve both performance and generalization to unseen tracks, with up to a 27% reduction in lap times compared to baseline models. Key hyperparameters such as batch size, epochs, learning rate, entropy, discount factor, loss type, and number of episodes are tuned to further optimize results. This leads to an additional 12% reduction in lap completion time and 35% variance improvement leading to faster convergence. Moreover, the models trained with optimized hyperparameters showed superior generalization to unseen tracks, outperforming baseline models. These findings offer valuable insights for improving reinforcement learning-based autonomous driving models, with potential applications in real-world autonomous systems.