A Hybrid Deep Learning Model for Pneumonia Diagnosis: Bridging Gaps in Imaging and Patient History Data

The use of multimodal data such as patient files and images is crucial for diagnosis in healthcare.In practice, we don't always have all patient history or images, and sometimes lab reports may be missing.This research suggests a Missing-Modality Resilient Model derived from causal inference and deep learning methodologies to counter this challenge by reconstructing missing modalities while maintaining reliable diagnosis.Using Kaggle chexpert X-ray data [6], CheXpert Plus Dataset [7], and composite synthetic patient history, models are trained and tested for large-scale and multimodal data.The provided model employs modality-specific encoders, CNN for X-rays and other tabular data for medical history.Additionally, the causal graph is designed with the help of the domain expert and statistical survey.This DAG represents the cause-effect structure of the X-ray features with patient attributes (age, smoking history, and so on).A modality reconstruction module can use Structural Causal Models (SCMs) and Variational Autoencoders (VAEs) to predict the missing data, so the features are realistic.These available and reconstructed modalities are combined and projected into a joint latent space for pneumonia diagnosis.Consistently, the performance loss is significantly outweighed by the proposed model when one or more modalities are absent, thus overcoming baseline methods.Furthermore, it brings improvements to the interpretability aspect through the representation of clinical variables' interactions using the causal inference framework.This work advances a corpus of missing data management for clinical applications by incorporating causal learning into flexible missing modality recovery to enhance diagnostic reliability across multiple modalities.This leaves a place for future work where the model could be employed for other illnesses apart from COPD.The performance of the model could be checked on more extensive data sets, and the model could be tested for its versatility when used in different healthcare facility settings.