QSD-STFormer: Quaternion Ship Domain-guided Spatiotemporal Transformer for ship trajectory prediction

With the rapid development of artificial intelligence, deep learning-based ship trajectory prediction has become essential for maritime collision avoidance and autonomous navigation. However, in ports and inland waterways, dense multi-ship interactions increase the uncertainty and diversity of ship motion, making trajectory prediction more challenging. To address this issue, this paper proposes a novel Quaternion Ship Domain-guided Spatiotemporal Transformer (QSD-STFormer) based ship trajectory prediction method. Grounding on the Conditional Variational Autoencoder (CVAE) framework, the method innovatively integrates physical navigation constraints: a QSD-guided Interaction Prior Module (QSD-IPM) generates physics-informed interaction weights to remedy pure data-driven modeling defects. A Spatiotemporal Joint Modeling (STJM) module then incorporates these weights into the attention mechanism to jointly extract temporal and spatial features, capturing dynamic ship interactions. Finally, a Gaussian Mixture Model-Latent Representation (GMM-LR) module models diverse navigation intentions, enhancing trajectory prediction accuracy and diversity. Experiments on real world AIS (Automatic Identification System) data from the Port of New York and the Wuhan Riverfront show that QSD-STFormer outperforms baselines in minADE, minFDE and MR. On the New York dataset, it reduces minADE by 15.28% and minFDE by 24.15% compared to the state-of-the-art baseline, demonstrating its effectiveness in improving trajectory prediction accuracy and autonomous navigation safety in navigation environments.