Machine learning for automated clinical coding in the neurology outpatients department

<h3>Introduction</h3> Clinical coding informs healthcare billing and planning, and is a complex, time consuming task. Existing automated models lack the accuracy to be used in real-life clinical systems. We explore the performance of machine learning clinical coding models on neurology outpatient encounters in a tertiary centre with an annual volume of 30 000 outpatients (80% not coded). <h3>Methods</h3> We used a training dataset of 14 000 neurology outpatient clinic letters , annotated with ICD-10 codes by neurologists. Data granularity was reduced by grouping diagnoses into 30 categories. Several Natural Language Processing (NLP) transformer models and one deep neural network model were trained. <h3>Results</h3> The top performing model, a DistilBERT transformer, achieved overall accuracy of 84.2% (weighted precision 81.9%, weighted F1-score 82.5%). At 0.9 prediction confidence threshold, 60% of letters were predicted with 95.5% accuracy. We deployed the model to audit 12-months of outpatient activity. Predictions under 0.6 confidence were annotated manually. The audit represents the first analysis of outpatients volume stratified by diagnosis in our department. <h3>Conclusion</h3> NLP models perform well when trained on speciality-specific text. While completely automated ICD- 10 coding remains unfeasible for now, lower granularity models can achieve good accuracy and significantly reduce the volume of manual coding.