Incomplete Multimodal Bone Tumor Image Classification Based on Attention Fusion and Feature Sharing

The classification of bone tumors (benign, intermediate, and malignant) is crucial for clinical treatment planning. Utilizing multimodal images such as X-ray, CT, and MRI for comprehensive assessment can improve the accuracy of bone tumor classification. However, in clinical practice, it is often challenging to collect images from all modalities, and the absence of certain modalities can lead to decreased classification accuracy. Existing methods typically involve extracting features from the available modalities to complete the missing modality features. Nevertheless, due to the differences between modalities, this approach can introduce feature completion bias, adversely affecting classification performance. Therefore, we propose an attention fusion and feature sharing module, which aggregates complementary features from existing modalities to learn similar shared features. These shared features are then used to represent the missing modality features, thereby reducing completion bias caused by modality differences and improving the accuracy of bone tumor classification. The proposed method was compared with 3 advanced methods for handling missing modalities. In 7 available modality combinations, the bone tumor classification performance of the proposed method significantly outperformed the other methods. Compared with the best existing method for handling missing modalities, the AUC value of the classification results of the proposed method increased by 2.3%, and the accuracy increased by 2.2%. This indicates that the proposed method exhibits excellent classification performance under various complex missing scenarios.