An AI-Driven Federated Framework for Distributed Multimodal Sensing with Implications for Loan Risk Prediction

Distributed multimodal sensor networks support agricultural monitoring and decision-making, but sensing nodes differ in data structure, sampling strategy, and environmental distribution, and centralized data pooling raises privacy and bandwidth concerns. We propose a Federated Multimodal Sensor Decision Network (FMSD-Net) that links multisource sensing to net-return prediction across distributed production sites. A Heterogeneous Multimodal Sensor Encoder (HMSE) encodes environmental time series, soil measurements, red–green–blue (RGB) canopy images, and structured management variables and fuses them with cross-modal multi-head attention. A non-IID-aware federated aggregation (NFA) rule weights client updates by distribution shift, update-direction consistency, and node reliability. A Sensor-to-Decision Prediction Module (SDPM) jointly predicts crop yield (t/ha), recorded production cost (103RMB/ha), and net return, defined as gross revenue minus recorded production cost (103RMB/ha), under an accounting-consistency constraint, and maps the predicted return-on-cost margin to a rule-based decision score. FMSD-Net was evaluated on three greenhouse tomato sites in Changping (Beijing), Baoding (Hebei), and Weifang (Shandong) using a chronological split by growing season (2022 training, 2023 validation, 2024 testing). On the pooled 2024 test set, FMSD-Net achieves a Mean Absolute Error (MAE) of 5.21×103RMB/ha, a Root Mean Squared Error (RMSE) of 7.36×103RMB/ha, a Mean Absolute Percentage Error (MAPE) of 8.12%, and an R2 of 0.927, reducing MAE by 5.1% relative to the strongest federated baseline, FedAWA. Under client-local evaluation, it achieves an MAE of 5.38×103RMB/ha and an R2 of 0.923 and reaches the convergence criterion in 56 communication rounds, 8.2% fewer than FedAWA, with 4.97 GB of total communication. A separate farmer–year loan panel provides complementary econometric evidence linking climate exposure to agricultural credit risk.