MMP-MSH: Multimodal Mortality Prediction Based on a Multilevel Semantic Hypergraph Network

Multimodal representation, as an application framework of social computing, enables researchers to utilize multimodal data for more accurate predictive analysis of mortality rates. In lengthy clinical texts, there are numerous neutral words that do not directly reflect the patient's condition. However, the prevalence of frequently occurring neutral words diminishes the weights of key terms that are directly relevant to the patient's condition, resulting in an imbalance in the allocation of text feature weights. To address this issue, we propose multimodal mortality prediction-multilevel semantic hypergraph (MMP-MSH), a medical multimodal model based on a multilevel semantic hypergraph. Specifically, we approach clinical text in two ways. First, we employ a CNN to extract textual features directly. Second, the text processed into hypergraph is subjected to multilevel GCN to obtain global hypergraph information, which is then introduced into the model training process and combined with the features of each batch of clinical texts. We conducted experiments on the MIMIC-III dataset to evaluate the effectiveness of MMP-MSH in predicting mortality rates.