NeRF-Enhanced Digital Twin for Building Anomaly Inspection Using Unmanned Aerial Systems (UASs)

Building facade inspections are crucial for ensuring structural integrity and occupant safety, traditionally requiring physical access that can be costly and risky. Recent advancements have seen the integration of Unmanned Aerial Systems (UASs) equipped with imaging technologies to facilitate these inspections. However, the primary challenge remains in the effective detection and precise localization of facade anomalies, such as cracks, stains, and specifically, thermal anomalies. This paper investigates the fusing of diverse data sources, specifically thermal infrared (IR) imaging and high-resolution RGB data, to enhance 3D registration of thermal anomalies. Utilizing a Computational Neural Radiance Field (NeRF) approach, it aims to reconstruct detailed 3D models of building facades. This integration improves the accuracy of anomaly detection by fusing the precise camera positions derived from the original RGB data to refine the alignment and visualization of thermal anamolies in both RGB and IR imagery. We utilized COLMAP for camera position estimation and NeRF for 3D registration, employing Structure-from-Motion (SfM) and NeRF methodologies to create detailed and scalable 3D models from 2D IR and RGB images. This approach integrates the precision of camera positioning with advanced 3D reconstruction techniques to enhance the visualization and analysis of building facades. Our method automates the registration and mapping of detected thermal anomalies in 2D images onto the reconstructed 3D models, improving the diagnostics of building facades by enabling precise localization and scaling of these anomalies. The findings demonstrate the potential of our approach to reduce inspection times and enhance the safety of diagnostic procedures through higher accuracy and less invasive methods.