Using Causal Attention Neural Networks for Social Media Analysis: Augmented Intelligence for Intervention in Online Discourse on Opioid Recovery

Social media platforms are important venues for people affected by opioid use disorders to seek support. To ensure a safe and respectful online environment, many social media platforms have implemented mechanisms involving human volunteers to moderate such discussions and enforce community rules, especially for complex topics involving marginalized populations. To enhance consistency and efficiency in moderation, we propose leveraging large language models to triage content and prioritize certain items for timely and proactive moderator review and intervention. Specifically, we develop a causal attention neural network (CANN) for the prediction of emotionally intense and harmful comments and simulate the moderation process. We demonstrate the superior performance of CANN against other algorithm benchmarks with a maximum relative improvement of 297.62% in area under the precision–recall (AUPR) curve and 12.61% in area under the receiver operating characteristic (AUROC) curve. CANN can be leveraged to obtain a 23.27% improvement in timely moderation.