A novel inflammatory bowel disease registry powered by artificial intelligence and natural language processing

Despite substantial recent progress in inflammatory bowel disease (IBD) therapies, there remains a large proportion of patients who are non-responders or develop significant adverse events. Accurate data registries may assist clinicians and researchers to gain insights into IBD and provide opportunities to improve overall patient care. Most data registries are limited by the amount of time and resources needed to collect and record patient-level data. We describe and validate a novel IBD repository (IBD Data Lake), leveraging machine learning and natural language processing (NLP) techniques, as useful tools to curate and retrieve pertinent, real-time clinical data in the IBD population. Structured and unstructured data were extracted from patients' electronic medical record and were transferred to a secure cloud infrastructure and curated into a searchable database. A customized user interface was created, and an advanced NLP service (Comprehend Medical) was employed to extract clinical information from unstructured data from medical documents. Chart review served as the gold standard for validation. Between July 1, 2018 and July 31, 2023, 208 (104 IBD and 104 matched non-IBD) patients were retrieved. The IBD Data Lake's performance metrics for identifying IBD patients were as follows: sensitivity 98.1%, specificity 97.1%, and accuracy 97.6%. The machine learning/NLP components demonstrated high performance in analyzing key IBD characteristics: for distinction of ulcerative colitis or Crohn's disease, sensitivity was 100% and specificity 98.2%; for smoking status, sensitivity was 100% and specificity 96.9%; and for extraintestinal manifestations, sensitivity was 92% and specificity 100%. A novel IBD Data Lake that integrates machine learning and NLP techniques has been established and validated. IBD patients were identified with high accuracy and the machine learning/NLP components enable comprehensive and timely data extraction. This will ultimately lay the groundwork for recruitment of specific IBD cohorts of interest to address the gaps that remain in our knowledge.