Collaborative Deep Q-Network for Multi-Robot Coverage Path Planning in Complex Environments

Abstract Existing multi-robot coverage methods in unknown environments suffer from inadequate use of inter-robot interaction information, limited collaborative decision-making capabilities, excessive repeated coverage, and insufficient adaptability to different team sizes and map environments. To address these issues, this paper proposes a collaborative deep Q-network (CD-DQN) framework. The framework constructs a task-state representation integrating robot-specific information, local environmental observations, regional task information, and inter-robot collaborative relationships. A multi-head collaborative attention module uses the current robot’s features as the query and the team members’ features as the keys and values to adaptively aggregate interaction information relevant to the current coverage decision. On this basis, action-value estimation using a dueling deep Q-network (Dueling DQN), dynamic region allocation, and a multi-level reward mechanism are combined to guide robots toward unexplored areas while reducing repeated coverage and motion conflicts. A fixed-length representation of collaborative features is also adopted, and the framework is trained with varying numbers of robots and randomly generated map layouts to improve policy adaptability to different team sizes and map environments. Experimental results show that, within the tested configurations, CD-DQN maintains high coverage performance and demonstrates adaptability to changes in map size and team size. Compared with the baseline algorithms, CD-DQN achieves higher coverage rates and fewer completion steps in both tested scenarios, with competitive redundancy and collaboration performance.