CottonCapT6: A Multi-Task Image Captioning Framework for Cotton Disease and Pest Diagnosis Using CrossViT and T5

The identification of cotton diseases and pests is crucial for maintaining cotton yield and quality. However, conventional manual methods are inefficient and prone to high error rates, limiting their practicality in real-world agricultural scenarios. Furthermore, Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) models are insufficient in generating fine-grained and semantically rich image captions, particularly for complex disease and pest features. To overcome these challenges, we introduce CottonCapT6, a novel multi-task image captioning framework based on the Cross Vision Transformer (CrossViT-18-Dagger-408) and Text-to-Text Transfer Transformer (T5). We also construct a new dataset containing annotated images of seven common cotton diseases and pests to support this work. Experimental results demonstrate that CottonCapT6 achieves a Consensus-based Image Captioning Evaluation (CIDEr) score of 197.2% on the captioning task, demonstrating outstanding performance. Notably, the framework excels in providing more descriptive, coherent, and contextually accurate captions. This approach has strong potential to be deployed in cotton farms in the future, helping pest control personnel and farmers make precise judgments on cotton diseases and pests. However, its generalizability to other crops and environmental conditions remains an area for future exploration.