A hybrid multi-modal framework for real-time crop recommendation via IoT sensors and weather API integration

This study presents a field-deployable crop recommendation system based on multi-modal data fusion. It integrates IoT sensor inputs (NPK, pH, temperature, humidity), deep learning-based soil image classification and real-time weather API data to provide site-specific recommendations. A machine learning ensemble processes the tabular data, a custom CNN classifies soil type, and their outputs are fused through a decision-mapping layer. A dual-mode user interface supports manual parameter entry and automated IoT data streams, allowing farmers to upload soil images, view sensor readings and receive recommendations in real time. Evaluation uses a source-photograph-disjoint partition of the image corpus and stratified ten-fold cross-validation of the tabular ensemble. On undegraded inputs the tabular ensemble alone attains 0.9954 ± 0.0041 weighted F1, so this benchmark admits no measurable accuracy gain from fusion, and none is claimed. The contribution of the multi-modal design is instead established under degradation: with sensor perturbation at nominal probe tolerance and field-realistic image degradation, the proposed late-fusion architecture retains 0.9331 weighted F1 against 0.9138 for the tabular ensemble alone and 0.7980 for early fusion. A block-permutation analysis quantifies the visual modality at 2–6 F1 points, a contribution that grows as sensor quality falls. End-to-end inference takes 51 ms on average.