Leveraging LLMs for ICD Coding and Uncertainty Estimation : Can the model's awareness of the hierarchical structureof ICD-10 codes impact its prediction performance?

Manual ICD coding of clinical notes with International Classificationof Diseases (ICD) codes is a time-consuming and error-prone task,yet critical for effective healthcare management, billing, and research.This thesis investigates the use of Large Language Models (LLMs) toautomate the prediction and assignment of ICD codes from clinicalnotes, aiming to enhance both the efficiency and accuracy of the pro-cess while reducing the reliance on human coders.We developed a novel approach to leverage the hierarchical struc-ture of ICD codes, which reflects the nested and interrelated natureof diagnoses, to improve the predictive capabilities of LLMs. By incor-porating this hierarchical awareness, the model is better equipped tocapture the dependencies among medical codes, thereby minimizingerrors associated with irrelevant or less likely disease classifications.Additionally, we implemented uncertainty estimation techniquesduring the inference phase to assess the confidence of the model’spredictions. By quantifying prediction uncertainty, we can identifycases where the model may be less certain, providing valuable in-sights into areas where human oversight might still be necessary.Our results show that integrating hierarchical structures reducesthe likelihood of predicting irrelevant diseases that are distant fromthe actual disease in the hierarchical tree, highlighting the potentialof this approach to improve ICD code assignment