CAD-assisted PointNet deep learning for autonomous bolt-loosening monitoring in steel structures

This study presents a novel approach for autonomous bolt-loosening monitoring by integrating computer-aided design (CAD)-assisted PointNet deep learning and three-dimensional (3D) point cloud processing. The following approaches are implemented to achieve the objective. First, a procedure of CAD-assisted databank generation for PointNet deep learning models is proposed. The CAD-generated databank contains diverse and well-labeled 3D point clouds of a bolt connection of steel girder with variation in loosening conditions. Second, PointNet segmentation and classification models are trained based on CAD-generated databank. The PointNet segmentation model is used to identify bolts from 3D point cloud, and PointNet classification models are used to identify bolt head angles and loosening lengths. Third, the CAD-assisted PointNet models are validated on real-world 3D point clouds of a steel girder bolt connection under various bolt-loosening levels. To enhance the performance of PointNet deep learning, advanced 3D point cloud processing techniques such as representative orientation alignment, bolt isolation using K -means clustering, and best-fitting rectangle for bolt group identification are introduced. Experimental results demonstrated that the proposed method effectively identifies bolt-loosening angles with high accuracy.