A hybrid transformer with spatiotemporal attention for citrus pest and disease forecasting

Accurate forecasting of citrus pests and diseases is of great significance for safeguarding orchard economic returns, reducing pesticide misuse, and promoting sustainable plant protection. However, the occurrence process of citrus pests and diseases exhibits pronounced spatiotemporal heterogeneity, strong nonlinearity, and coupling among multi-source environmental factors, which makes it difficult for traditional machine learning methods and single-architecture deep learning models to simultaneously characterize long-range temporal evolution and small-scale spatial diffusion. To address these issues, this paper proposes ST-CitrusFormer, a hybrid Transformer with spatiotemporal attention, for multi-step forecasting of the occurrence and severity of multiple citrus pests and diseases. The model adopts an encoder–decoder architecture and integrates multi-scale convolutional feature extraction, a sliding-window temporal self-attention branch, a spatial graph-attention branch built upon a graph weighted by both geographical distance and management similarity, and a gated fusion module. A Focal–Huber joint loss is further designed to alleviate the long-tailed class imbalance problem. Based on a multi-source spatiotemporal dataset comprising approximately 18,720 samples with 14-dimensional features, independently collected from 12 sampling plots in a single typical citrus demonstration orchard in southern China during 2019–2024, comparative experiments were conducted on four target pests and diseases: Huanglongbing (HLB), citrus red mite, aphid, and sooty mold. On the chronologically held-out 2024 test set, ST-CitrusFormer achieved an average F1-score of 0.812, exceeding LSTM, Informer, PatchTST, and the strongest recent baseline, TimeXer, by 7.9, 3.5, 2.0, and 1.1 percentage points (pp), respectively. Its 7-day-ahead RMSE on the same test set was 5.2% lower than that of TimeXer and 11.6% lower than that of PatchTST. Ablation, hyperparameter-sensitivity, attention-visualisation, leave-one-block, and leave-one-year analyses further supported the effectiveness, interpretability, and within-orchard robustness of the model. These findings position ST-CitrusFormer as a promising framework for fine-scale forecasting within the studied orchard; however, multi-site, multi-variety, and multi-climate validation remains necessary before operational deployment.