Metafusion: A Novel Method for Integrating Clinical Metadata with Imaging Modalities for Medical Applications

Clinical history is the primary determinant of medical diagnosis and must be utilized with imaging data for accurate identification of medical conditions. We propose MetaFusion, a novel method for multi-modal data fusion that inte-grates clinical data with imaging data to improve the quality of diagnosis across a range of diseases using three distinct datasets. We demonstrate significant improvements over prior methods in skin cancer type detection using camera images, breast condition detection via mammograms, and glau-coma detection through retinal fundus images. Specifically, we observe improvements of 3.5% in balanced accuracy for skin cancer type detection, 9.7% for breast condition detection, and 7.1% for glaucoma detection compared to the top-performing prior methods. Our code is available at: https://github.com/ananyar1007/MetaFusion