An Immersive Digital Twin Framework for Robotic-based Nondestructive Evaluation of Civil Infrastructures

This study introduces a novel digital twin (DT)-based framework for robotic-based non-destructive evaluation (NDE) of civil structures, focusing on real-time data integration and the immersive visualization of inspections. The proposed framework consists of four components: 1) a physical robotic arm equipped with an NDE measurement device, 2) a database for storing and managing NDE data, 3) a game engine supporting virtual reality (VR) simulations, and 4) VR hardware for user interaction. Utilizing the Federal Highway Administration’s Virtual NDE Laboratory (vLab) as a case study, the framework demonstrates the integration of as-built drawings with 2D and 3D NDE data, such as laser surface profiles and high-resolution imaging (HRI). The study specifically evaluates six NDE methods: impact echo (IE), ultrasonic surface waves (USW), infrared thermography (IRT), ground-penetrating radar (GPR), electrical resistivity (ER), and HRI, achieving visualization of sub-millimeter defects at resolutions up to 1440 x 1584 pixels and refresh rate of 72 Hz enabling low-latency, real-time data collection and visualization. The results show the framework facilitates interactive visualization of NDE maps as layers that can be viewed individually or overlayed on as-built drawings and other NDE layers for comparison. By enabling spatial alignment of the robotic platform in the immersive VR environment, the framework also facilitates remote inspection and two-way interaction for data collection and visualization. As a demonstrative case, the results visualize this capability in remote operation of a laser surface profiler (LSP).