A two-stage data filtering framework for on-the-go proximal soil sensing data to improve digital mapping of soil properties

• A two-stage framework combines rule-based screening and DBSCAN outlier detection. • KDE-based DBSCAN parameterisation avoids subjective manual tuning. • Severe interference from subsurface metal pipelines was detected and removed. • Mean LOOCV RMSE for ordinary kriging EC a maps decreased by 67% across depths. • Soil texture prediction improved, with sand R 2 increasing from 0.62 to 0.76. On-the-go proximal soil sensing systems mounted on vehicles can rapidly acquire high-density soil information for precision agriculture. However, raw measurements are often affected by factors such as warm-up drift, attitude changes, abrupt turns, and microtopography, degrading data quality and the reliability of subsequent soil property maps. In this study, a two-stage data filtering framework was developed to improve on-the-go apparent electrical conductivity (EC a ) measured at four depths (0.54, 1.03, 1.55, and 3.18 m) using a DUALEM-21S electromagnetic induction sensor. Stage 1 applied physically interpretable, rule-based screening to remove gross outliers. Stage 2 encompassed density-based spatial clustering of applications with noise (DBSCAN) in a time-space median-deviation feature space to detect residual outliers, with parameters derived automatically using kernel density estimation. In a 41-ha field dataset, 39% of raw records were removed as erroneous, primarily due to noise from metal pipelines and headland turns. The mean leave-one-out cross-validation RMSE for ordinary kriging interpolation maps, averaged across the four depths, decreased by 67%. Soil texture prediction was also improved using filtered data. Specifically, for a partial least squares regression (PLSR) model predicting surface sand content, R² increased from 0.62 to 0.76, and RMSE decreased from 6.90% to 5.52%. These results demonstrate that the proposed framework enhanced the spatial consistency of on-the-go EC a data and the accuracy of derived soil maps. The framework provides an objective, portable, and computationally efficient protocol for field-scale implementation that can be generalized to filter most high-density proximal soil-sensing data collected during mobile sensor operations.