Heterogeneous agent proximal policy optimization for UAVs-USV formation navigation with multi-modal perception

As the foundation of air-sea integration, heterogeneous collaboration holds significant potential for various maritime applications. However, existing research often decomposes the formation collaborative navigation decision-making (FCND) problem into isolated sub-tasks, lacking a unified framework that simultaneously addresses formation maintenance, multi-agent cooperation, and collision avoidance. To bridge this gap, this paper proposes a novel FCND strategy for unmanned aerial vehicles-unmanned surface vehicle (UAVs-USV) systems based on the heterogeneous agent proximal policy optimization (HAPPO) algorithm. The main contributions are threefold: (1) An advantage decomposition mechanism and symmetric policy update scheme are introduced, enabling HAPPO to generate diverse action spaces with minimal redundancy according to heterogeneous characteristics. (2) A joint state space is constructed by integrating continuous visual data and LiDAR observations, with a multimodal feature fusion network applied to extract shared representations and quantify intra-modal affinity. (3) A comprehensive reward function is designed, incorporating camera modeling, velocity prediction, and collaboration error, thus balancing formation maintenance with collaborative performance. Simulation results demonstrate that the proposed FCND strategy outperforms existing methods across multiple evaluation metrics. Physical experiments, serving as a proof of concept, further validate the deploy ability of the strategy in real-world scenarios, providing critical insights for practical engineering applications. • We propose a novel UAVs-USV formation collaboration navigation decision-making strategy based on HAPPO algorithm. • Establish advantage decomposition and symmetric policy updates to ensure HAPPO outputs diverse action spaces with minimal redundancy. • The multimodal feature fusion network is employed to extract shared features specific and quantify the affinity within each modality.