Optimizing Token Fusion Mechanisms in Swin Transformer for Improved Feature Representation and Fine-Grained Insect Classification

Effective pest monitoring is critical for ensuring global food security, as insect pests pose significant threats to crop yields and agricultural sustainability. Traditional pest identification methods rely on manual inspection, which is time-consuming, labor-intensive, and susceptible to human error. Deep learning, particularly Convolutional Neural Networks (CNNs), has been widely applied to automate insect classification; however, these models exhibit limitations in capturing long-range dependencies and hierarchical feature representations. Transformer-based architectures, such as Swin Transformer, have demonstrated improved performance in vision tasks, yet their effectiveness in fine-grained insect classification remains constrained by cross-window communication inefficiencies and suboptimal hierarchical feature aggregation. This study proposes an enhanced Swin Transformer model incorporating a Token Fusion mechanism to address these limitations. The proposed Cross-Window Token Fusion (CWTF) facilitates feature propagation across non-overlapping windows, while Hierarchical Token Fusion (HTF) enhances multi-scale representation learning. Experimental evaluations on the IP102 dataset, a large-scale benchmark for insect classification, demonstrate that the proposed model outperforms CNN-based and standard Transformer-based models in classification accuracy while maintaining computational efficiency. The findings highlight Token Fusions effectiveness in strengthening feature interactions, thereby advancing the development of highly accurate and computationally efficient automated pest monitoring systems. This research contributes to the broader adoption of Transformer-based architectures in precision agriculture, enhancing real-time pest surveillance and sustainable crop management.