A critical review of advances in diffuse reflectance spectroscopy for rapid soil fertility assessment

Abstract The growing pressure on soil resources from climate change, land degradation, and the increasing demand for food security has intensified the need for rapid, accurate, and cost-effective methods of soil characterisation. Diffuse reflectance spectroscopy (DRS), operating across the visible-near infrared (Vis-NIR: 350–2500 nm) and mid-infrared (MIR: 4000–400 cm⁻¹) regions of the electromagnetic spectrum, has emerged as a powerful analytical platform that addresses these challenges. By exploiting the diagnostic absorption features arising from molecular vibrations and electronic transitions of key soil chromophores (organic matter, clay minerals, iron oxides, calcium carbonates, and water), DRS enables the simultaneous estimation of numerous soil properties from a single spectral scan, without generating chemical waste and with minimal sample preparation. This review synthesises more than nine decades of scientific progress in soil spectroscopy, from the early spectral libraries of the 1930s to the contemporary integration of machine-learning and deep-learning frameworks. We examine the mechanisms underlying spectral absorption, critically assess the comparative performance of Vis-NIR and MIR techniques, evaluate pre-processing strategies from Savitzky-Golay smoothing and standard normal variate correction to derivative transformations, and benchmark the full spectrum of prediction models, from partial least squares regression (PLSR) and multivariate adaptive regression splines (MARS) to support vector regression (SVR), random forests, and convolutional neural networks. The review also critically documents performance across a comprehensive range of soil properties including texture, organic carbon, cation exchange capacity, pH, electrical conductivity, and macro- and micronutrients, highlighting consistent achievements as well as persistent limitations. A central argument of this review is that while the field has achieved remarkable predictive capabilities for certain core properties, key assumptions regarding model transferability, chromophore linearity, and pre-processing universality remain empirically untested. The paper concludes by identifying the five most consequential unanswered research questions in the discipline, with particular attention to the challenges posed by the heterogeneous agro-ecological landscapes of India. Recommendations for a methodologically rigorous path forward are provided. To make the comparison more transparent, selected validation results are aggregated by fertility property rather than pooled across incompatible studies. The comparison supports selective use of DRS for clay or texture, organic matter, cation-exchange capacity, and total nitrogen, while electrical conductivity, available phosphorus, potassium, and DTPA-extractable micronutrients remain dependent on the calibration domain and validation design.