Enhancing Autonomous Multi-Agent Coordination for Unmanned Ground Vehicle Operations in Denied Urban Environments Using Proximal Policy Optimization

This paper explores the use of multi-agent reinforcement learning (MARL), specifically Proximal Policy Optimization (PPO), to improve the coordination and exploration efficiency of autonomous unmanned ground vehicles (UGVs) operating in contested urban environments. Motivated by the challenges of modern urban conflicts-highlighted by recent experiences such as the ongoing conflict in Ukraine-this study seeks to enhance autonomous decision-making and adaptability where communication networks may be compromised or unreliable, and hostile agents may be present. Building upon previous work comparing heuristic and RL-based exploration methods, this manuscript details the iterative refinement of PPO through reward shaping, revisit decay penalties, and training parameter adjustments. Results demonstrate that PPO significantly improves exploration coverage, stability, and efficiency, though some challenges in policy generalization and convergence stability remain. The study concludes by identifying future research directions, including real-world hardware integration and enhanced scalability methods for multi-agent deployments. Although adversarial entities and communication constraints are not explicitly modeled in the current simulations, the reinforcement learning framework developed here is extensible to contested and denied operational environments.