VIOS-Net: A Multi-Task Fusion System for Maritime Surveillance Through Visible and Infrared Imaging

Automatic ship monitoring models leveraging image recognition have become integral to regulatory applications within maritime management, with multi-source image co-monitoring serving as the primary method for achieving comprehensive, round-the-clock surveillance. Despite their widespread use, the existing models predominantly train each data source independently or simultaneously train multiple sources without fully optimizing the integration of similar information. This approach, while capable of all-weather detection, results in the underutilization of data features from related sources and unnecessary repetition in model training, leading to excessive time consumption. To address these inefficiencies, this paper introduces a novel multi-task learning framework designed to enhance the utilization of data features from diverse information sources, thereby reducing training time, lowering costs, and improving recognition accuracy. The proposed model, VIOS-Net, integrates the advantages of both visible and infrared data sources to meet the challenges of all-weather, all-day ship monitoring under complex environmental conditions. VIOS-Net employs a Shared Bottom network architecture, utilizing both shared and specific feature extraction modules at the model’s lower and upper layers, respectively, to optimize the system’s recognition capabilities and maximize data utilization efficiency. The experimental results demonstrate that VIOS-Net achieves an accuracy of 96.20% across both visible and infrared spectral datasets, significantly outperforming the baseline ResNet-34 model, which attained accuracies of only 4.86% and 9.04% in visible and infrared data, respectively. Moreover, VIOS-Net reduces the number of parameters by 48.82% compared to the baseline, achieving optimal performance in multi-spectral ship monitoring. Extensive ablation studies further validate the effectiveness of the individual modules within the proposed framework.