Benchmark Accuracy Is Not Field Reliability: Auditing Crop and Fertilizer Recommendation Datasets and Their Robustness to Sensor Noise

Machine-learning crop and fertilizer recommenders are usually judged on clean public benchmarks, where accuracy above 95% is routinely reported. We test whether such scores reflect field reliability. On the Crop Recommendation dataset (2,200 samples, 22 classes), seven classifiers reach 97.1-99.6% under repeated stratified cross-validation, with Random Forest best. However, when Gaussian measurement noise of 0.25 feature standard deviations is added, Random Forest falls to 82.0% while SVM and KNN retain 92.9% and 92.3%, so the model ranking changes. On the Fertilizer Prediction dataset (100,000 samples, 7 classes) every model performs at chance (about 14.3%); statistical tests show the labels are independent of the features. We contribute a dataset-audit procedure, a noise-robustness protocol, and an open web prototype that warns when a model performs at chance.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24333947
Primary Topic
Smart Agriculture and AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Benchmark Accuracy Is Not Field Reliability: Auditing Crop and Fertilizer Recommendation Datasets and Their Robustness to Sensor Noise

Debjit Das, Anshumaan Das
4 links
Zenodo (CERN European Organization for Nuclear Research)
Smart Agriculture and AI
preprint

Benchmark Accuracy Is Not Field Reliability: Auditing Crop and Fertilizer Recommendation Datasets and Their Robustness to Sensor Noise

Debjit Das, Anshumaan Das
preprint en

Abstract

Machine-learning crop and fertilizer recommenders are usually judged on clean public benchmarks, where accuracy above 95% is routinely reported. We test whether such scores reflect field reliability. On the Crop Recommendation dataset (2,200 samples, 22 classes), seven classifiers reach 97.1-99.6% under repeated stratified cross-validation, with Random Forest best. However, when Gaussian measurement noise of 0.25 feature standard deviations is added, Random Forest falls to 82.0% while SVM and KNN retain 92.9% and 92.3%, so the model ranking changes. On the Fertilizer Prediction dataset (100,000 samples, 7 classes) every model performs at chance (about 14.3%); statistical tests show the labels are independent of the features. We contribute a dataset-audit procedure, a noise-robustness protocol, and an open web prototype that warns when a model performs at chance.

Zenodo (CERN European Organization for Nuclear Research)
SRM Institute of Science and Technology (IN)
Zero hunger
Smart Agriculture and AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon