DIMMC: A 3D vision approach for structural displacement measurement using a moving camera

Structural displacement is an essential data for engineers to evaluate the health and safety of civil structures. Computer vision-based displacement measurement has become increasingly popular due to accessibility and cost-effectiveness. While most of the studies in this area have been focused on analysing footage taken by static cameras, the usage of moving cameras such as those mounted on Unmanned Aerial Vehicles (UAVs) presents a transformative opportunity for structural health monitoring (SHM). Moving cameras enable flexible and scalable displacement measurement, allowing inspections in hard-to-reach areas without the constraints of fixed installation points. This paper proposes a deep learning-based, purely computer vision system that requires no auxiliary devices, for measuring structural displacement using monocular footage from a single moving camera. This system utilizes a Vision Transformer (ViT)-based mesh deformation neural network to reconstruct 3D geometry from a single image. State-of-the-art deep learning models for object segmentation and tracking are also used to isolate relevant objects from the background. To convert the reconstructed coordinate system into real-world coordinates, a 3D point registration method is introduced. The system eliminates the need for installing artificial targets or detecting key points on the structure, enhancing the robustness of the measurement process. The effectiveness and accuracy of the proposed approach are verified through experimental tests on beam-shaped structures, with cross-correlation coefficients exceeding 0.97 and mean absolute percentage errors remaining below 33 % for dynamic test. These findings underscore the accuracy of the proposed approach and its promising potential for practical applications. • A purely vision-based system using a monocular moving camera for structural displacement measurement. • Eliminated the need for artificial targets or key point detection. • Utilized a Transformer based neural network to reconstruct 3D structural geometry from single images . • Enhanced 3D reconstruction accuracy with large vision deep learning models for object segmentation and tracking. • Validated on beam structures with Cross-Correlation larger than 0.97 and MAPE smaller than 33 %.