A novel method for soil organic carbon prediction using integrated ‘ground-air-space’ multimodal remote sensing data

• Integration of multi-modal remote sensing data from ground-air-space. • A cross-scale and cross-hierarchy data fusion method, ResoCroS-Net, was developed. • Leverage UAV data as a baseline layer for mapping soil organic carbon content. • Advanced spatial and spectral downscaling algorithms are proposed based on deep learning. High-accuracy maps of soil organic carbon (SOC) content are essential for agricultural management and ecosystem services. However, traditional remote sensing data can hardly balance spatial and spectral resolution, resulting in the inability to simultaneously obtain high-resolution spatial distribution of SOC and detailed spectral features, thus restricting the comprehensive resolution of soil detail information. To address this challenge, we propose an innovative multimodal remote sensing data integration framework, ResoCroS-Net, which integrates ground (soil samples), air (unmanned aerial vehicle (UAV) images), and space (satellite images) data, realized effective integration of multimodal data at different resolutions, with particular innovations in hierarchical design and data-processing logic. Specifically, ground samples and air images are combined to generate high accuracy SOC maps, which serve as the ’spatial’ baseline for ResoCroS-Net used as a reference for subsequent high-resolution image generation. Next, low spatial resolution images (ZY1-02D, Sentinel-2A) are downscaled to high spatial resolution images using Enhanced Super-Resolution Generative Adversarial Network models within adversarial networks. Meanwhile, typical spectral features in ZY1-02D satellite data were extracted, and a spectral dictionary was constructed for accurate reduction of details in low spectral resolution images. Using spectral unmixing networks and sparse representation networks, low spectral resolution images are downscaled to high spectral resolution. Then, joint spatial-spectral features were extracted by 3D convolutional neural network. Based on this framework, we developed Model (i), which integrates SOC data with spatial-spectral resolution downscaling (SSD) image; Model (ii), which integrates SOC data, UAV image with spatial resolution downscaling (SD) image; and Model (iii), which integrates SOC data, UAV image with SSD image. We also evaluated the performance of various algorithms (e.g., Random Forest (RF), Convolutional Neural Networks (CNN), Graph Neural Networks (GNN), and Multi-Layer Perceptron (MLP)) across these models. The SOC prediction experiments conducted in Youyi, the largest state farm in China, demonstrate that Model (iii) achieves the highest accuracy with the GNN model. This model improves coefficient of determination (R 2 ) and ratio of performance to interquartile distance (RPIQ) by 0.09 and 0.28, respectively, and reduces root mean square error (RMSE) by 0.52 g kg −1 compared to Model (ii). In addition, using UAV data as the baseline layer significantly improves prediction accuracy, with R 2 and RPIQ increasing by 0.17 and 0.58, respectively, and RMSE decreasing by 1.09g kg −1 . Regarding model performance in SOC content prediction, GNN is more suitable for Model (i) and Model (iii), while CNN is more appropriate for Model (ii). In conclusion, ResoCroS-Net achieved collaborative optimization of spatial-spectral features across scales, provided a significant advantage in improving the accuracy of quantitative remote sensing, offers an efficient and accurate SOC monitoring tool, providing a practical basis for the integrated remote sensing theory of ‘ground-air-space’.