Enhancing Vessel Situational Awareness Through Multimodal Fusion of Radar Images and AIS Data

With the development of waterway transportation, single-sensor systems face limitations such as incomplete and inaccurate perception of surrounding vessel targets in complex maritime environments, posing new challenges and requirements for vessel situational awareness. This paper proposes a multi-modal maritime target vessel perception framework, which employs deep neural networks to fuse radar images, high-precision positioning data from ARPA messages, and vessel identity information from AIS, constructing a cross-modal feature alignment model. Using ARPA messages as spatiotemporal reference labels, a nonlinear mapping relationship between radar pixel coordinates and geographic coordinates is established. The results demonstrate that the collaborative prediction of multi-modal data achieves higher accuracy and better alignment with AIS trajectories. The proposed multi-modal data-based target vessel situational awareness solution enables more comprehensive perception of target vessel states, thereby enhancing navigation safety. It holds significant application potential in operational scenarios such as vessel collision avoidance, maritime surveillance, and early warning systems.