More than the Sum of its Parts: Applying Topic Modeling and Explainable AI to Deep Learning in Understanding Problematic Opioid Use

Problematic opioid use is a major crisis, especially among Veterans of the U.S. Military. In previous work we developed a natural language processing tool to identify problematic opioid use in Veterans Affairs clinical notes. In this work, we developed an application that identifies clinical notes associated with patients who were documented as experiencing problematic opioid use only in clinical notes, as compared to patients who had received a relevant ICD code for problematic opioid use. The application used topic and topic word output from topic modeling to train deep neural network models. The models performed well, achieving area under the curve values of 82% or above, exceeding the performance of baseline models using only topics or topic words. We also computed impact scores, an explainable artificial intelligence method that identifies critical features in training deep learning models. The impact scores extended additional understanding to the data and outcomes.