MMGC-Net: Deep neural network for classification of mineral grains using multi-modal polarization images

The multi-modal characteristics of mineral particles play a pivotal role in enhancing the classification accuracy, which is critical for obtaining a profound understanding of the Earth's composition and ensuring effective exploitation utilization of its resources. However, the existing methods for classifying mineral particles do not fully utilize these multi-modal features, thereby limiting the classification accuracy. Furthermore, when conventional multi-modal image classification methods are applied to plane-polarized and cross-polarized sequence images of mineral particles, they encounter issues such as information loss, misaligned features, and challenges in spatiotemporal feature extraction. To address these challenges, we propose a multi-modal mineral particle polarization image classification network (MMGC-Net) for precise mineral particle classification. Initially, MMGC-Net employs a two-dimensional (2D) backbone network with shared parameters to extract features from two types of polarized images to ensure feature alignment. Subsequently, a cross-polarized intra-modal feature fusion module is designed to refine the spatiotemporal features from the extracted features of the cross-polarized sequence images. Ultimately, the inter-modal feature fusion module integrates the two types of modal features to enhance the classification precision. Quantitative and qualitative experimental results indicate that when compared with the current state-of-the-art multi-modal image classification methods, MMGC-Net demonstrates marked superiority in terms of mineral particle multi-modal feature learning and four classification evaluation metrics. It also demonstrates better stability than the existing models.