Condition assessment of bolted connections in steel structures using deep learning

Abstract Steel structure connections are prone to bolt loosening and subsequent loss of strength or stability if not inspected periodically. Use of noncontact sensing and artificial intelligence can substantially increase the safety and efficiency of these inspections; however, the existing defect detection models do not account for variability of defects in both missing and loosened bolt connections. Furthermore, the performance of available deep learning models can be substantially diminished due to the presence of background noise in images of real structures. Finding these defects could be even more difficult in highway steel structures due to the complex and continuously changing nature of background. Additionally, no datasets are available to properly use these models for defect detection. In this paper, Mask RCNN (MRCN), You Only Look Once (YOLO), and Faster RCNN (FRCNN) models were adapted for investigation. These models were trained and tuned on a unique dataset-developed by the Authors comprised of 1100 raw images with connection defects of laboratory-made and in-service structures. A series of image enhancement and augmentation techniques were used to superimpose bolt and nut defects on intact images of in-service ancillary structure and to increase the size in training dataset to 1500. After models achieved satisfactory performance in laboratory images, they were tested on field and lab data (including 200 images with defective bolt). Thermal images were used to develop an efficient way to mask image background in visual images. The result indicated that removing background decreased the false positive rate (FRP) around 65% between the investigated deep learning models on UAS image. Overall accuracy of these models estimated between where 90% and 93%when FRCNN and Mask RCN.