Robust 3D vessel trajectory monitoring with monocular vision based deep learning and surveillance cameras

Vision-based vessel trajectory monitoring systems (VTMSs) have gained prominence in intelligence navigation, enhancing navigation safety and operational efficiency through accurate monitoring, route planning, and early collision risk detection. However, current systems face challenges. Most rely on deep neural networks (DNNs) with tracking-by-detection, which only provide rough vessel positions using bounding boxes, leading to imprecise trajectory estimation. These systems typically assume ideal visibility conditions and struggle in adverse weather conditions such as rain or fog, where visibility is reduced. Additionally, these systems typically focus on 2D trajectories and require multiple cameras or costly equipment for 3D reconstruction, complicating setups and limiting comprehensive tracking. This paper presents a monocular vision-based VTMS that uses a single static camera for vessel tracking and 3D trajectory reconstruction. The system detects vessels using an object detection network and refines their contours with a segmentation foundation model, improving tracking accuracy. To enhance performance in foggy conditions, it applies a dark channel prior-based fog removal algorithm. Vessel movements are mapped from 2D to real-world 3D coordinates using homography-based transformations, eliminating the need for multiple cameras or depth sensors. Extensive experiments on river surveillance videos under clear, foggy, and nighttime conditions demonstrate the system's robustness and reliability. The object detection network achieves the average precision (AP IoU:50 % ) and average recall (AR IoU:50–95 % ) exceeding 80 % on the testing dataset. By incorporating the segmentation model, the tracking error is significantly reduced, yielding smoother trajectories with fewer outliers and improved temporal consistency compared to traditional tracking-by-detection methods. This improvement is observed in both 2D and 3D trajectories across various vessel videos captured from a real-world river, including those taken in foggy weather conditions and night conditions. This system offers a cost-effective and scalable solution for vessel monitoring in rivers, harbors, and coastal areas, with potential applications in maritime traffic management and bridge-vessel collision detection to enhance navigation safety. • A monocular vision method for ship trajectory monitoring based on surveillance cameras. • Advanced computer vision techniques for ship detection and segmentation. • Accurate and robust ship detection in extreme conditions such as rainy, foggy and night.