Transformer in civil engineering defect detection: A survey

Detecting structural and functional defects in large-scale civil infrastructure during operation is of paramount importance, and employing intelligent algorithms for detection holds significant value. Deep learning technology emerges as the primary avenue to accomplish this task. Recently, Transformer self-attention models have garnered attention as alternatives to deep convolutional neural networks due to their robust parallel computing capabilities and adeptness in modeling long-range dependencies. Harnessing this paradigm shift, the field of civil engineering has delved into exploring Transformer's applicability in intelligent defect detection tasks. Motivated by this, we conducted a systematic investigation into the latest engineering defect detection applications of Transformers. Our survey encompasses over 40 engineering detection algorithms based on Transformers, with roadways, tunnels structures, and bridges serving as primary application scenarios. Lastly, we discuss key challenges encountered in these applications and provide insights into future development directions. This survey marks a systematic overview of Transformer applications in the domain of civil engineering. Its objective is to aid researchers in grasping the concept and architecture of Transformer models, and to furnish suitable reference algorithms and application strategies for intelligent detection research in civil infrastructure.