Federated Reinforcement Learning and Deep Q-Network: Improving Fault Tolerance and Energy Consumption in Swarm Robotics for Mine Prospection Missions

This article focuses on improving fault tolerance and optimizing energy consumption in the context of a mining prospection mission conducted by a swarm of autonomous robots. Two major contributions are proposed. The first aims to reduce communication between robots in order to increase the system’s robustness in the presence of failures. The second focuses on minimizing the trajectory of a deminer robot to reduce overall energy consumption. To address these goals, two reinforcement learning-based algorithms are proposed: Deep Q-Network (DQN) and Federated Reinforcement Learning (FRL), both derived from the Q-learning algorithm. Simulation results examining the impact of the exploration rate α on the number of detected mines show that, with 10 autonomous robots of the same architecture and 30 randomly placed mines over 30 experiments, the FRL algorithm provides better fault tolerance and ensures that the main prospection mission is accomplished even in the presence of some robotic failures or errors. Furthermore, a second series of 60 experiments involving the integration of the deminer robot, focused on optimizing energy consumption, demonstrates that the DQN algorithm is more effective in reducing energy usage, due to improved a better optimization of unnecessary deminer movements, while successfully resolving deadlock situations that the latter may encounter. These findings open the door to the development of a hybrid algorithm combining the strengths of DQN and FRL to ensure both system robustness and minimal energy consumption.