Multiview attention networks for fine-grained watershed categorization via knowledge distillation

With the rapid development of artificial intelligence technology, an increasing number of village-related modeling problems have been addressed. However, first, the exploration of village-related watershed fine-grained classification problems, particularly the multi-view watershed fine-grained classification problem, has been hindered by dataset collection limitations; Second, village-related modeling networks typically employ convolutional modules for attentional modeling to extract salient features, yet they lack global attentional feature modeling capabilities; Lastly, the extensive number of parameters and significant computational demands render village-related watershed fine-grained classification networks infeasible for end-device deployment. To tackle these challenges, we introduce a multi-view attention mechanism designed for precise watershed classification, leveraging knowledge distillation techniques, abbreviated as MANet-KD. Specifically, first, we have developed the inaugural multi-view watershed classification dataset, termed MVWD.Second, we introduce a cross-view attention module (CVAM), which models salient features from intersecting views with global attention, enhancing the accuracy and precision of watershed classification. This module enhances fine-grained classification accuracy. Based on the above proposed CVAM, we propose a heavyweight MANet-Teacher and a lightweight MANet-Student, and finally, we introduce an Attention Knowledge Distillation (AKD) strategy that effectively transfers critical feature knowledge from the teacher network to the student network, utilizing the AKD approach for enhanced learning outcomes. The experimental results show that the proposed MANet-Teacher achieves state-of-the-art performance with 78.51% accuracy, and the proposed MANet-Student achieves comparable performance to MANet-Teacher with 6.64M parameters and 1.68G computation. The proposed MANet-KD achieves a good balance of performance and efficiency in the multi-view fine-grained watershed classification task. To facilitate further research in multi-view fine-grained watershed classification, all datasets, codes, and benchmark outcomes will be made available to the public. https://github.com/Jack13026212687/MANet-KD.