Density-aware Adaptive Outlier Rejection for a Point-to-plane Measurement-model-based LiDAR-inertial Odometry

LiDAR-Inertial odometry (LIO) is affected by outliers arising from data-association errors, dynamic objects, and unstable feature extraction that can degrade estimation performance and increase long-term drifts. In point-to-plane registration LIO such as FAST-LIO2, outlier handling is often implemented by applying a single global threshold to a point-to-plane distance. This approach has two key limitations: (i) it does not account for the fact that measurement uncertainty varies with the prior estimate and environment, and (ii) it treats the reliability of local plane estimation as spatially uniform within a scan, despite strong local variations. To address these limitations, we propose an adaptive outlier rejection method developed based on two complementary mechanisms. First, we employ innovation gating for uncertainty-aware outlier rejection. Specifically, measurement residuals are normalized by the innovation covariance. Second, we propose density-aware measurement classification, which uses local point density as a measure of plane-estimation reliability. The density-based classification applies class-wise thresholds to reflect heterogeneous geometric conditions within a scan. Experiments on both simulation and open datasets demonstrate that the proposed method reduces errors in pose estimates compared with the conventional approach.