C5LS: An Enhanced YOLOv8-Based Model for Detecting Densely Distributed Small Insulators in Complex Railway Environments

The complex environment along railway lines, characterized by low imaging quality, strong background interference, and densely distributed small objects, causes existing detection models to suffer from low accuracy in practical applications. To tackle these challenges, this study aims to develop a robust and lightweight insulator detection model specifically optimized for these challenging railway scenarios. To this end, we release a dedicated comprehensive dataset named complexRailway that covers typical railway scenarios to address the limitations of existing insulator datasets, such as the lack of small-scale objects in high-interference backgrounds. On this basis, we present CutP5-LargeKernelAttention-SIoU (C5LS), an improved YOLOv8 variant with three key improvements: (1) optimized YOLOv8’s detection head by removing the P5 branch to improve feature extraction for small- and medium-sized targets while reducing computational redundancy, (2) integrating a lightweight Large Separable Kernel Attention (LSKA) module to expand the receptive field and improve contextual modeling, (3) and replacing CIoU with SIoU loss to refine localization accuracy and accelerate convergence. Experimental results demonstrate that it reaches 94.7% in [email protected] and 65.5% in [email protected]–0.95, outperforming the baseline model by 1.9% and 3.5%, respectively. With an inference speed of 104 FPS and a model size of 13.9 MB, the model balances high precision and lightweight deployment. By providing stable and accurate insulator detection, C5LS not only offers reliable spatial positioning basis for subsequent defect identification but also builds an efficient and feasible intelligent monitoring solution for these failure-prone insulators, thereby effectively enhancing the operational safety and maintenance efficiency of the railway power system.