SITS-BERT ++: Enhancing Satellite Image Time Series Analysis with Novel Architectural and Methodological Innovations

In this study, we introduce SITS-BERT++, a novel adaptation of the SITS-BERT model tailored for the analysis of annual satellite observation time series data. Our approach integrates the Swish activation function and a sophisticated time warping data augmentation technique to significantly en-hance model performance. Additionally, we propose a multi-faceted architectural overhaul that includes dual activation functions-Swish for the overall model and GeL U for specific prediction and classification tasks-resulting in a substantial improvement in accuracy from 90.61 % to 93.07%. These inno-vations demonstrate the efficacy of SITS-BERT ++ in advancing the state-of-the-art in satellite time series analysis, surpassing the SITS analysis models.