Simulating Physical Climate Models Using Temporal Transformers and Spatial Data as Computational Shortcuts

This paper presents a transformer-based deep learning model to simulate physical climate models. To address its complexity and statical, we propose a novel approach which combines spatial features extracted from satellite remote sensing images and climate time series data as a more accurate reflection of real-world dynamics. Our model extracts spatial features for each timestamp, generates embedding layers for each spatial feature vector, and consolidates these embedding vectors into the encoding layer of the Transformer. The self-attention mechanism is also applied to the decoding layer with the mask attention removed, which aims to reduce the model complexity. Compared to other time series models, our work achieves better performance in simulating physical climate models. This model features an image input to image output design, aiming to assist non-expert researchers in climate rapidly predicting future climate conditions. Our work shows the potential of applying deep learning to simplify physical models in the field of climate, thereby opening up new possibilities for satellite remote sensing image research and forecasting related climate conditions.