Position-Guided Hybrid Convolutional Neural Network and Transformer Network for steel strip surface defect detection

Surface defect detection is crucial for maintaining the quality standards of steel strip products. However, the irregular shapes, varying sizes, and background similarity of defect targets pose significant challenges for precise identification and localization using existing methods. To address these issues, this paper presents a Position-guided Hybrid Convolutional Neural Network (CNN) and Transformer Network based on the You Only Look Once version 5 small (YOLOv5s) model (PCT-Net). This innovative approach leverages the strengths of both CNN and Transformer architectures to comprehensively analyze complex semantic information of defective targets from both local and global perspectives. The proposed PCT-Net introduces three key components: First, a novel hybrid CNN and Transformer benchmark module named CNN and Transformer Block (CTBlock) is proposed and used to construct an efficient backbone feature extraction network, CNN and Transformer Network (CTNet). This network ensures accurate defect characterization, even for small defects that closely resemble the background. Second, an Attention-Based Local and Global Feature Fusion Module (ALGM) is designed and integrated into the neck network to effectively combine multi-scale features at both local and global levels, enhancing local details while establishing global contextual relationships. Third, a Target Position-Guided Module (TPGM) is proposed and integrated into the detection head to focus the model’s attention on defect target regions, enabling precise defect localization. To overcome the scarcity of surface defect data in industrial production, we developed the Xsteel Surface Defect Detection Dataset (XSDD-DET). Extensive experiments conducted on three steel strip surface defect datasets, North Eastern University Surface Defect Detection Dataset (NEU-DET), Global 10 Class Metallic Surface Defect Detection Dataset (GC10-DET), and XSDD-DET, demonstrate the superior performance of PCT-Net, achieving mean Average Precision (mAP) values of 80.7%, 69.2%, and 94.5%, respectively. The model maintains real-time inference at 51.1 Frames Per Second (FPS), exceeding the 30 FPS threshold required for industrial deployment. These results underscore the exceptional capability of PCT-Net in steel strip surface defect detection, significantly advancing the field of industrial quality control. Code is available at https://github.com/MYF1124/PCT-Net .