Real-Time Rust Segmentation on Steel Surface Using a Lightweight Transformer Deep Learning Model

Identifying rust on steel surfaces is crucial for preserving the integrity of civil infrastructure. Rust-induced damage poses a significant challenge to structural health monitoring systems. Timely and accurate monitoring of rust defects on steel surfaces is essential to ensure the structural integrity and longevity of these assets. Traditional rust identification, especially for steel bridges, is labor-intensive, time-consuming, and poses a labor hazard. The purpose of this research is to develop a lightweight transformer deep-learning model called SegFormer for near-real-time rust segmentation on steel surfaces. The proposed model is a simple, efficient, and powerful solution for semantic segmentation, which utilizes transformers and lightweight multilayer perceptron decoders to reduce processing time and enhance accuracy. Traditional deep learning models based on convolutional architectures require positional encoding, which can lead to reduced performance when the resolution of the test images differs. In contrast, SegFormer, a hybrid architecture that integrates components from transformers and neural networks, eliminates the need for positional encoding. By utilizing a local-global data processing system, SegFormer has the potential to enhance segmentation accuracy. While most real-time models focus on rust detection, detection alone is insufficient for a comprehensive analysis of defects. In this study, we introduce a novel method for identifying rust through segmentation techniques, enabling near-real-time analysis. This research utilizes the PyTorch Lighting backend to effectively decrease the processing time and provide a rust recognition approach that operates in almost real-time. It can rapidly analyze rust segmentation at the individual pixel level in images, allowing for a swift assessment of the damage caused by rust on steel structures. The model was trained on a data set of annotated steel surface images. The proposed model obtained an overall accuracy of 96.67% and an image processing interface rate of 157 frames per second (fps), significantly reducing processing time for unseen images. The proposed model obtained a precision of 95.67%, recall of 95.63%, accuracy of 95.63%, and an image processing interface rate of 157 fps, significantly reducing processing time for unseen images. Additionally, a high area under the curve of 0.92 indicates that the model’s accuracy is robust. These results suggest that the proposed method is highly suitable for near-real-time rust segmentation.