YOLOv11 with transformer attention for real-time monitoring of ships: A federated learning approach for maritime surveillance

• Proposed YOLOv11 with Transformer Attention to enhance detection accuracy for ship monitoring in real-time maritime environments. • Integrated Transformer-based attention modules into YOLOv11 for better global feature representation of maritime targets. • Employed a federated learning framework to train the detection model across distributed edge devices without sharing raw data, preserving privacy and bandwidth. • Achieved significant improvements in precision, recall, and mAP over state- of -the -art models on a benchmark ship dataset. • Validated the system’s real-time inference capability and robustness, making it suitable for deployment in coastal surveillance networks. Maritime surveillance plays a very important role in marine navigation, control and enforcement. It is necessary for real time monitoring of illicit activities and regulating maritime traffic. Satellite images of ships and vessels pose difficulty in identifying ships because of different object sizes, occlusions, and intricate backdrops. In addition, traditional centralized machine learning models for detection of ships in satellite images often depend on centralized architectures, which lead to considerable communication overhead and to the creation of a single point of failure. This paper proposes a novel technique that combines Federated Learning (FL) with YOLOv11, empowered with a transformer-based attention module, to improve ship detection accuracy in real time while ensuring data confidentiality. The federated architecture allows for distributed and collaborative training of a global model, thus securing raw data, preserving privacy, and addressing data heterogeneity. The combination of cutting-edge object detection ability in real time offered by YOLOv11 with a transformer-based attention module helps in better feature extraction and spatial context understanding, which is essential for identifying small or camouflaged ships in a convoluted maritime environment. The suggested method was evaluated on benchmark satellite imagery dataset, exhibiting a marked enhancement in detection accuracy, especially for small and partially obscured ships. The proposed method achieved an accuracy of 95 percent, a recall of 90 percent, mAP50 of 92 percent and mAP50-95 of 54.7 percent. The model also attains real-time inference speeds, thus demonstrating the applicability of the methodology for scalable and secure maritime surveillance applications. Comparative experiments display that our methodology exceeds current cutting-edge approaches in accuracy, precision, recall, and mean average precision (mAP), while sustaining computational efficiency. This research accentuates the potential of hybrid YOLO-transformer frameworks for enhancing real-time object detection in complex FL settings.