Transformer-based global attention for long-range rail surface defect detection

Detection of rail surface discrete defects (RSDDs) is critical for train safety. Convolutional neural networks (CNNs) are the first choice because of their excellent feature learning ability. However, we note that CNNs are difficult to capture global information of a rail surface image due to limitation of local receptive fields. To solve this problem, we propose a novel transformer-based global attention for long-range rail surface defect detection network, called TGLNet. Specifically, a rail surface image is first divided into non-overlapping patches and then projected into embedding space, which denoted as tokens. They are processed by a global feature interaction (GFI) module. Afterwards, tokens are divided into multiple windows, and window partitions are shifted to generate new windows, enabling interactions between windows and allowing TGLNet to focus on global and local features of a rail surface image. Finally, the feature information is used to accurately identify and localize RSDDs. We have evaluated models on the public datasets. Experimental results show that the performance of TGLNet is better than baseline models.