Consensus Based Reinforcement Learning Method for Differentiated Formation Control of UAVs

Obstacle avoidance is the crux of formation control for a UAV swarm, no matter in urban or wild application environments. But there is a dilemma to promote both of obstacle avoidance flexibility and consistency in UAV formation control, which has not been solved effectively by existing researches. In view of this, this paper proposes a consensus based reinforcement learning method for differentiated formation control (DFC) of UAVs, to balance the obstacle avoidance flexibility and consistency. In this method, to promote obstacle avoidance flexibility of a UAV swarm, a consensus mechanism with differentiated formation control strategies is designed, it allows each UAV in a swarm changes its formation control strategy among aggregation, formation keeping and obstacle avoidance according to its local environment, and calculates its own current subgoal based on the selected strategy. Further, to improve the flight efficiency of the UAV swarm, a reinforcement learning model is provided to generate the optimal offset vector for each UAV according to its current subgoal. Moreover, to enhance the obstacle avoidance consistency of the UAV swarm, a collaborative obstacle avoidance algorithm is designed in the obstacle avoidance strategy, it requires UAVs to share their obstacle information and obstacle avoidance actions, and provides obstacle avoidance consensus rules to help UAVs to choose consistent obstacle avoidance directions. The experiment results show that the proposed method can combine obstacle avoidance flexibility and consistency of UAVs, thereby achieving higher flight efficiency and maintaining stable network connectivity.