Hybrid Atmospheric Modeling of Refractive Index Gradients in Long-Range TLS-Based Deformation Monitoring

Terrestrial laser scanners (TLS) are widely used for deformation monitoring due to their ability to rapidly generate 3D point clouds. However, high-precision deliverables are increasingly required in TLS-based remote sensing applications to distinguish between measurement accuracies and actual geometric displacements. This study addresses the impact of atmospheric refraction, a primary source of systematic error in long-range terrestrial laser scanning, which causes laser beams to deviate from their theoretical path and intersect different object points on the target surface. A comprehensive study of two physical refractive index models (Ciddor and Closed Formula) is presented here, along with further developments on 3D spatial gradients of the refractive index. Field experiments were conducted using two long-range terrestrial laser scanners (Leica ScanStation P50 (Leica Geosystems, Heerbrugg, Switzerland) and Maptek I-Site 8820 (Maptek, Adelaide, Australia)) with reference back to a control network at two monitoring sites: a mine site for long-range measurements and a dam site for vertical angle measurements. The results demonstrate that, while conventional physical atmospheric models provide moderate improvement in accuracy, typically at the centimeter- or millimeter-level, the proposed advanced physical model—incorporating refractive index gradients—and the hybrid physical model—combining validated field results from the advanced model with a neural network algorithm—consistently achieve reliable millimeter-level accuracy in 3D point coordinates, by explicitly accounting for refractive index variations along the laser path. The robustness of these findings was further confirmed across different scanners and scanning environments.