Take your time: Using Natural Language Processing on Safety Reports to Reduce Incident Likelihood

Abstract Workplace injuries carry significant social and financial consequences for both employees and employers. Through Behavior-Based Safety (BBS), workplaces both identify at-risk behaviors to reduce injuries and their consequences while also encouraging individuals to recognize and reinforce safe behaviors that promote safer practices (Ludwig & Laske, 2022). Most companies already use this process; however, there are often problems with the quality of these reports, aka ‘pencil whipping’ (marking reports blank or lacking any meaningful information). We aim to determine if there is a connection between the quality of safety reporting and incident likelihood, and what dimensions of ‘quality’ have the most impact on reducing incident likelihood. We hypothesize that reports with higher quality descriptions that are specific, actionable, and clear will show stronger associations with reduced incident risk compared to more vague reports. Methods Our analysis will be using three years of safety data from a manufacturing plant in the southwest region of the United States, taken from 2022 to 2024. We will select units with significant safety variance and reporting, then use text analysis to identify ‘pencil whipping’ by examining whether a safety report lacks a description or uses minimal wording (e.g., marking only “safe” or “unsafe”), which would be scored as a low-quality report. Using a natural language processing model (NLP), reports that are determined to be higher quality will be analyzed further by assessing length, specificity, actionability and clarity (Kjell et al., 2023). These ratings will then be used in a hierarchical regression model to understand the effect of each quality measure on incident and near-miss likelihood. Expected Results Because previous literature suggests that certain quality factors contribute to the effectiveness of behavioral checklists (Leslie et al., 2021), higher-quality reports are expected to reduce incident and near-miss likelihood. Specifically, reports that provide adequate actionable detail and high contextual detail are expected to significantly reduce the likelihood of safety incidents and near misses, whereas reports with vague or generic context are expected to show weaker effects. The quality of safety reporting on safety incidents is important for practitioners because it will inform safety training. Organizations that work towards increasing the quality of their safety reporting, can work towards reducing incidents, creating a safer workplace for all.