Enhancing ICD-9 Code Prediction with BERT: A Multi-Label Classification Approach Using MIMIC-III Clinical Data

This study addresses the challenge of accurately predicting ICD-9 codes from clinical notes in the MIMIC-III database, focusing on intensive care unit (ICU) records. Existing methods using the MIMIC-III dataset, such as CAML, have achieved significant results using convolutional neural networks. In this paper, we leverage BERT (Bidirectional Encoder Representations from Transformers) to enhance the multi-label classification of ICD-9 codes from clinical text. We preprocess the MIMIC-III dataset to address the sequence length constraints imposed by BERT, employing techniques like automatic truncation and section-based filtering to optimize input data. Our model utilizes the BERT architecture fine-tuned on the top 50 ICD-9 codes to improve prediction accuracy, employing binary cross-entropy for multi-label classification. Experimental results show that our approach achieves competitive performance, with a Micro-F1 score of 0.83 after a single epoch using sequences of length 128. Future work will explore using longer sequence lengths and compare performance with existing CAML pre-processing methods. This study demonstrates the potential of BERT in advancing automated ICD-9 code assignment in clinical settings, providing a robust framework for future improvements in healthcare informatics.