Learning-Based Multi-Stage Formation Scheduling with a Hybrid Controller

In the past decades, multi-agent systems have been a hot topic due to their wide applications, and the formation of multi-agent systems is a branch involving navigation, obstacle avoidance, controller design, and other issues. Due to the increasing requirements for accuracy and efficiency, as well as for a bridge to link the sensing and control stages, the importance of transmission scheduling is gradually emerging, with the scheduling of limited resources under various constraints to better complete tasks becoming a focus of attention. However, most of the literature only considers the formation process as a whole, while overlooking the discrepancies in the formation process at different time stages. In this paper, a multi-stage formation scheduling problem with limited communication resources is studied. A multi-stage model has been proposed based on the different completion levels of formation. Compared to the single-stage model, the proposed multi-stage model reflects the different requirements during the formation process. Furthermore, in order to save communication energy, three transmission modes have been defined to reduce energy consumption in terms of communication frequency and communication radius. Considering the need for the dynamic scheduling of coupled parameters, we propose a reinforcement-learning-based hybrid controller which includes a basic controller and a fuzzy controller. The hybrid controller, which continuously adjusts parameters according to the requirements of each stage, conducts a trade-off between system performance and energy consumption. Additionally, the reinforcement learning ensures that all parameters are optimal in the corresponding situation. The simulation results show that our controller ensures both dynamic and steady-state performance with lower energy consumption. The comparison with other scheduling strategies demonstrates the optimality and effectiveness of our proposed framework and algorithms.