Dynamic Reward Shaping with State-Action Feedback for UAV Control

The application of reinforcement learning in UAV control has made significant progress, especially in autonomous navigation and task execution in complex environments. However, traditional reward design and algorithms often face issues such as sparse rewards and low training efficiency. To address these challenges, this paper proposes an improved method for UAV heading control tasks. The proposed State-Action Based Reward Augmentation (SARA) algorithm utilizes state and action information from historical trajectories to precisely evaluate the impact of each decision point on task performance, while incorporating Beta distribution to provide additional reward signals to the agent. A dynamic trajectory clearing mechanism is introduced to improve the real-time calculation of rewards and reduce computational overhead. Additionally, the k-nearest neighbors (KNN) method is used to dynamically adjust the Beta distribution parameters, further enhancing the accuracy of the reward signals. These improvements not only enhance the performance of the algorithm in UAV tasks but also provide new insights into the application of reinforcement learning in control tasks.