Bone Fracture Detection via GANs-Based Multi-Modal Fusion Technique

Diagnosing fractures from medical images is challenging due to the limited availability of large, annotated datasets and the inherent variability across imaging modalities, such as CT and X-ray. Generating synthetic images that combine information from different modalities may enhance diagnostic accuracy. In this work, we propose a GAN-based multi-modal image fusion framework to generate synthetic X-ray images from CT scans. The generated images were evaluated both qualitatively and quantitatively by comparing them with real Xrays using metrics such as MSE, PSNR, and SSIM. To evaluate the effectiveness of the fused data, we trained a ResNet-18 classifier to differentiate between fractured and non-fractured knees, incrementally augmenting the original image with additional fused channels. The results showed a clear improvement in classification performance when fused modalities were included, particularly when two or three fusion outputs were combined. This approach demonstrates significant potential for advancing diagnostic tools in medical imaging, particularly when multimodal data is limited or unpaired.