An approach to classifying and measuring stigmatizing and positive language at scale in electronic health records

Prior studies have demonstrated that clinicians’ attitudes towards patients can manifest in the language they use in clinical notes, and that this language can impact the attitudes of other clinicians reading those notes and the quality of care patients receive. To facilitate large-scale analysis of such language in the electronic health record (EHR), we developed an approach to identifying and semantically classifying stigmatizing and positive EHR language using natural language processing (NLP) methods. We first developed a dictionary of stigmatizing and positive terms used in clinicians’ notes and sorted them into semantic categories. We annotated 10,244 examples of those terms within clinical notes from 2 academic health systems, according to whether they conveyed stigmatizing or positive sentiments. We used these annotated examples to train and validate NLP models to identify and semantically categorize stigmatizing and positive EHR language. We ran our NLP models on a separate set of notes from a single health system, written between 2016 and 2023 in one of 4 clinical departments: Emergency Medicine, Internal Medicine, Obstetrics & Gynecology, and Surgery (including surgical subspecialties). We examined the prevalence of different semantic categories of EHR language, overall and across clinical departments, and used novel methods to evaluate the impact of note templates and copy-forward practices on language prevalence. Our dictionary included terms in 4 semantic categories: credibility, demeanor, conformity with recommended care, and physical appearance. Across 12,841,577 notes, stigmatizing terms related to patient conformity (e.g., noncompliant) were most common (4.23% of notes), followed by those suggesting the patient lacked credibility (0.86%) or had a challenging demeanor (0.82%). Stigmatizing terms were copied forward in up to 25% of notes; positive terms (e.g., pleasant) were commonly included in templates (up to 65% of notes). Both stigmatizing and positive terms varied in prevalence across clinical contexts. Stigmatizing and positive EHR language conveys different sentiments that vary in prevalence and reflect different underlying biases. Our methods can be used to measure EHR language as markers of institutional culture and bias, guide interventions to target the specific types of bias identified as most problematic, and monitor the impact of those interventions.