Optimising healthcare process efficiency with HyperBERT: embedding ICD code hierarchy for national healthcare systems simulation and improvement

This research study focuses on embedding the hierarchical structure of the International Classification of Diseases (ICD) codes for automatic code assignment in clinical texts. It explores: 1) incorporating diagnosis code hierarchy as auxiliary knowledge in a BERT training process; 2) how transformer-based models like BERT can serve as interpretable AI in Ethiopia's National Classification of Diseases (NCOD) and the MIMIC dataset. Utilising qualitative methods, including experimental and case study strategies, results indicate that a new model architecture called HyperBERT improved the macro average AUC by 0.003 and F1-score by 0.02. The model's interpretability was demonstrated through a code-wise label attention mechanism using the bidirectional encoder representation for transformer (BERT) visualisation tool. Riemannian optimisation and Poincaré loss were employed to validate a new document representation model combining BERT with hyperbolic embeddings, enabling effective ICD code predictions from discharge summaries in Ethiopia's healthcare system.