Optimizing agricultural classification with masked image modeling

Image classification poses a significant challenge in agriculture. However, the utilization of popular algorithms such as vision transformers and convolutional neural networks has often fallen short in numerous agricultural classification tasks owing to the scarcity of extensively labelled data and the reliance on pretrained models trained on generic datasets. To address this, our study details the pretraining of ViTs using 224,228 agricultural images, employing masked image modeling for preprocessing. The pretrained model was then fine-tuned on three independent agricultural classification datasets and performed better than state-of-the-art methods. For example, our method achieved the highest accuracy rates of 76.18%, 98.49%, and 88.56% on IP102, DeepWeeds, and Tsinghua Dogs datasets, respectively. This enhancement in accuracy can be attributed to the robust modeling strategy we have developed through extensive experimentation with the pretrained MIM model. Our modeling strategy encompasses employing advanced MIM models, leveraging the histogram of oriented gradient features as the reconstruction target, and selecting an appropriate mask ratio. We hope that this research will prompt the application of self-supervised learning techniques, represented by the MIM model, to a wide range of agricultural image-related tasks in the future.