A mechanism scheme for improving extremist content detection & moderation algorithms

Algorithms are increasingly deployed as a frontline defense against digital extremism, yet their true effectiveness remains poorly understood. This review analyzes a decade of research (2013–2023) on algorithms for content detection and moderation. While researchers have reported high effectiveness for detection tools, we find this research constrained by a reliance on limited, platform-specific datasets. Beyond assessing current tools, we propose a mechanism scheme for using algorithms to interrupt the radicalization process. We conclude by outlining a new research agenda, calling for transparent, collaborative partnerships between social scientists and engineers to better detect and moderate extremist content and to ensure that the people who own the code are held accountable for the digital reality they create. To implement this agenda, social scientists should lead independent audits of moderation outcomes, while engineers should integrate accountability by embedding design principles into the early stages of algorithmic development. Furthermore, funders should mandate public–private transparency agreements as a prerequisite for research and development support.