Pixel-level image localization for updating 3D digital twins of dams using frequency convolutional networks

This paper presents PIL3D, an automated pixel-level image localization framework for maintaining up-to-date 3D digital twins of large-scale civil infrastructure, with a focus on dam structures. Unlike conventional 3D model updating approaches that require extensive manual data acquisition and labor-intensive processing, PIL3D automatically predicts the 3D coordinates of every pixel in an input image relative to an existing model, enabling fully automated dense pixel-to-point correspondences. Experimental validation on a real-world dam case demonstrates centimeter-level localization accuracy, significantly reducing manual intervention, data collection requirements, and computational demand. By integrating PIL3D into digital twin workflows, infrastructure inspection, monitoring, and maintenance can be streamlined into a continuous, automated process, advancing the state of automation in construction and asset management. • Deep learning method for pixel-level image localization within a 3D model. • Convolutional neural network leveraging frequency features for improved accuracy. • Training based on rendered images and depth maps instead of real data. • Feasibility demonstrated for automating 3D model updates.