GoM-ICD: Automatic ICD Coding with Gap Schemes and Mixture of Experts

Assigning standardized International Classification of Disease (ICD) codes to Electronic Medical Records (EMR) is crucial for enhancing the efficiency and accuracy of medical coding processes. However, existing methods face challenges in effectively capturing, integrating, and amalgamating specialized medical knowledge from complex textual data. In this study, we propose GoM-ICD, an advanced automatic ICD coding framework that integrates multiple gap schemes with a Mixture of Experts (MoE) architecture. GoM-ICD is designed to address the extreme multilabel text classification in ICD coding. It segments and reassembles text to facilitate seamless information exchange across different chunks, employing various segmentation methods derived from different gap schemes. Then the model-level MoE consolidates the predictions of these methods to enhance the prediction performance. Specifically, the segmented text is input to a Pretrained Language Model (PLM) to extract textual features. Next, a Bidirectional Long Short-Term Memory network (BiLSTM) is employed to capture long-term contextual information from the textual features. Finally, a text-level MoE, followed by a label-level MoE, enables precise attention matching between text and labels, thereby improving the fidelity of the coding process. The three levels of MoE leverage the collective insights of diverse expert models, effectively processing multi-dimensional text features and unifying model-level insights from various gap schemes. Extensive experimental results demonstrate that GoM-ICD achieves the state-of-the-art performance in automatic ICD coding tasks, reaching micro-F1 of 0.617, 0.620, and 0.613 on datasets MIMIC-III full, MIMIC-III clean, and MIMIC-IV ICD-10, respectively. The source code can be obtained from https://github.com/CSUBioGroup/GoM-ICD.