MS-LIP: Multiscale Semantic Information Integration With Large Language Models for Marine Prediction

Marine environmental forecasting presents a significant challenge in geoscience and remote sensing applications, particularly when modeling the complex multi-scale dynamics evident in satellite-derived sea surface temperature (SST) data. While recent deep learning approaches have shown promise in processing remote sensing data, they often fail to effectively capture the rich semantic information inherent in multi-scale oceanic phenomena, resulting in suboptimal prediction performance. To address this limitation, we propose MS-LIP, a novel neural network framework that leverages large language models (LLMs) to integrate multi-scale semantic information from satellite data for enhanced marine element prediction, with primary focus on SST forecasting. Our key innovation, the Multi-Scale Semantic Information Extraction Module (MSIE), employs cross-modal attention mechanisms to dynamically align LLM-derived semantic representations with remotely sensed spatiotemporal features at multiple scales, enabling comprehensive capture of thermal patterns from localized anomalies to basin-wide circulations. Additionally, we introduce the Amplitude-Phase Decoupling Interaction Module (APDI), which performs frequency-domain analysis specifically designed for satellite-derived SST data, extracting deeper insights from thermal patterns that conventional methods typically overlook. These components are integrated into our Domain-Specific Fine-Tuning for Pre-trained Large Language Models (DFPM) framework, which incorporates an efficient Low-Rank Adaptation with Mixture of Experts (LoRA-MoE) strategy tailored to marine remote sensing applications. Experimental validation using satellite-derived NOAA OISST data demonstrates state-of-the-art SST prediction performance, with error reductions of 7.12%, 5.88%, and 2.40% for temperature ranges around 15°C, 20°C, and 25°C, respectively. Further experiments on ERA5 reanalysis data for significant wave height (SWH) prediction demonstrate the framework’s generalizability to other marine elements, highlighting its versatility and potential for broad application in marine environmental forecasting.