Classification of arsenic contamination in soil across the EU by vis-NIR spectroscopy and machine learning

• A continental-scale framework for classifying soil As contamination was proposed. • Framework was determined by detailed model selection and hyperparameter tuning. • Accurate identification of high/low As contamination was achieved continentally. • Model interpretation revealed the response of As contamination to soil properties. Detecting soil arsenic (As) contamination is crucial for designing efficient soil remediation strategies; however, traditional laboratory-based As detection techniques are time- and labour-intensive and are unsuitable for large-scale spatial analyses. To address this issue, we combined machine learning (ML) with visible-near-infrared (vis-NIR) spectroscopy to develop an efficient framework for As detection in soil. The optimal spectral preprocessing method was determined, and eight ML models were compared. The support vector classifier achieved optimal performance after subsequent hyperparameter tuning, with area under the curve (AUC) and accuracy values of 0.89 and 0.83, respectively. Important spectral bands at 471 and 2422 nm were identified by permutation importance and correspond to Fe-oxide and carbonate, respectively. These two wavelengths were included in the partial dependence plot (PDP), revealing that the likelihood of soil As contamination decreased with increasing reflectance at wavelengths of 471 and 2422 nm due to a decrease in Fe-oxide and carbonate content. Consistent with this finding, two-way PDP analysis revealed that the As content of soil increased with increasing Fe-oxide and carbonate content. The model’s classification performance was further improved using an ensemble technique based on three optimal ML models, resulting in increased AUC and accuracy values of 0.9 and 0.83, respectively. Overall, the framework presented in this study enabled the precise classification of soil As content at the continental scale, while also indirectly explained the complex relationships between As content and soil properties.