Toward Patient-Specific Digital Twin Models of Disease Progression Using Sequential Medical Imaging and EHR Data

Artificial intelligence (AI) is reshaping healthcare by supporting faster and more informed clinical decisions. However, the complexity of human health makes accurate predictive modeling challenging. In this study, we introduce a methodological framework for constructing intelligent digital twins of disease progression by combining patients’ sequential medical images with temporally aligned electronic health records (EHRs). EHRs in this context include structured clinical parameters such as laboratory test results, demographic characteristics, and medication information. The existing literature provides limited approaches that jointly forecast future medical images and clinical status using long-term historical data. Our framework integrates aligned temporal image sequences with these EHR features and employs either ConvLSTM or ViViT-based spatio-temporal encoders, optionally coupled with a generative module for future image synthesis. While awaiting access to patient datasets, we conducted an initial evaluation using a single-cell time-lapse microscopy dataset whose temporal dynamics resemble patient data. Both systems generate time-ordered image sequences that evolve under changing conditions, and the shifting nutrient environment in microfluidic channels parallels the temporal variations observed in patients’ EHR records. This preliminary study demonstrates the broader applicability of our model to datasets containing long-term sequential images and associated parameters, supporting its potential for future patient-specific digital twin development.