DMF-ED: a dual-modal fusion model for emergency department service demand forecasting

Overcrowding in the emergency department causes patient flow problems with serious consequences for the entire healthcare system. Accurate forecasting of emergency department service demand can help medical service planners make resource allocation decisions, so as to improve the operational efficiency and service quality of hospitals. However, most of the existing studies are only limited to utilizing the temporal numerical modality of emergency department service demand data for forecasting. The joint analysis and effective fusion modeling between the different representation modalities of the service demand time series in the emergency department has not been fully explored yet. In this study, we first convert emergency department service demand into visual representations to explore the potential characteristics of the two-dimensional image patterns of the time series of emergency department service demands. Then, a multi-scale convolutional neural network model that fuses the original temporal numerical modality and its imaging modality is proposed to forecast the time series of emergency department service demand. Furthermore, to ensure multi-modal consistency and enhance the complementarity of multi-modal information fusion, the cross-attention mechanism and a co-training strategy are introduced to strengthen the advantages of information fusion and overcome the semantic inconsistency of representation in heterogeneous modality fusion. We conduct comparative experiments, ablation experiments, significance test and robustness analysis of the proposed model. Compared with the existing baseline models, the proposed model significantly reduces the forecasting error. Compared with the existing baseline model, the proposed model significantly reduces the prediction error. The reliability of the model’s forecasts and the necessity of each component of the model are verified. The empirical results highlight the potential of multi-modal information fusion in improving the accuracy of emergency department service demand forecasting.