PPO-CIS : A deep reinforcement learning framework for real-time toxicity detection in social media

• Introduces PPO-CIS, a novel Deep Reinforcement Learning (DRL) framework for adaptive toxicity detection in classifier cascades. • Utilizes Proximal Policy Optimization (PPO) to dynamically select classifiers based on sample complexity and system cost constraints. • Proposes a custom reward function balancing classification accuracy, latency, and computational efficiency. • Demonstrates improved performance over static cascades and individual models across two benchmark datasets: Kaggle and ToxiGen. • Achieves significant gains in throughput and accuracy, making the system suitable for real-time content moderation at scale. • Provides a scalable and cost-effective moderation solution for social media platforms facing high data volume and regulatory pressure. Online platforms face growing challenges in moderating harmful user-generated content due to the large volume and rapid pace of interactions. In existing moderation systems, automated tools assist human moderators, yet they often struggle to balance processing efficiency and reliable classification. When moderation fails to detect harmful content quickly and accurately, platforms risk user harm and noncompliance with content safety regulations. This paper proposes an adaptive moderation method named the Proximal Policy Optimization-based Cascaded Inference System (PPO-CIS). The method integrates multiple toxicity classifiers into a cascaded decision architecture guided by deep reinforcement learning. At each step, PPO-CIS selects the next classifier according to the content difficulty and the expected gain in accuracy relative to computational cost. The system enables rapid filtering of benign content and only activates high-capacity models for uncertain cases. PPO-CIS is the first toxicity detection framework to employ Proximal Policy Optimization for real-time optimization of classifier cascades. Experiments on the AugmenToxic and ToxiGen datasets show that PPO-CIS improves detection accuracy by 2.10 percent while increasing processing speed from 42.74 to 384 samples per second compared with static cascade designs. The findings show that adaptive model selection can better shield users from exposure to harmful content while lowering moderation costs. PPO-CIS provides a practical solution for deploying scalable and timely content moderation in fast-moving online environments.

Authors

Institutions

Publication Details

Journal
Knowledge-Based Systems
Published
2026-03-07
DOI
https://doi.org/10.1016/j.knosys.2026.115704
Primary Topic
Hate Speech and Cyberbullying Detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

PPO-CIS : A deep reinforcement learning framework for real-time toxicity detection in social media

Arezo Bodaghi, Ketra Schmitt, Benjamin C.M. Fung
Knowledge-Based Systems
Hate Speech and Cyberbullying Detection
article

PPO-CIS : A deep reinforcement learning framework for real-time toxicity detection in social media

Arezo Bodaghi, Ketra Schmitt, Benjamin C.M. Fung
article en

Abstract

• Introduces PPO-CIS, a novel Deep Reinforcement Learning (DRL) framework for adaptive toxicity detection in classifier cascades. • Utilizes Proximal Policy Optimization (PPO) to dynamically select classifiers based on sample complexity and system cost constraints. • Proposes a custom reward function balancing classification accuracy, latency, and computational efficiency. • Demonstrates improved performance over static cascades and individual models across two benchmark datasets: Kaggle and ToxiGen. • Achieves significant gains in throughput and accuracy, making the system suitable for real-time content moderation at scale. • Provides a scalable and cost-effective moderation solution for social media platforms facing high data volume and regulatory pressure. Online platforms face growing challenges in moderating harmful user-generated content due to the large volume and rapid pace of interactions. In existing moderation systems, automated tools assist human moderators, yet they often struggle to balance processing efficiency and reliable classification. When moderation fails to detect harmful content quickly and accurately, platforms risk user harm and noncompliance with content safety regulations. This paper proposes an adaptive moderation method named the Proximal Policy Optimization-based Cascaded Inference System (PPO-CIS). The method integrates multiple toxicity classifiers into a cascaded decision architecture guided by deep reinforcement learning. At each step, PPO-CIS selects the next classifier according to the content difficulty and the expected gain in accuracy relative to computational cost. The system enables rapid filtering of benign content and only activates high-capacity models for uncertain cases. PPO-CIS is the first toxicity detection framework to employ Proximal Policy Optimization for real-time optimization of classifier cascades. Experiments on the AugmenToxic and ToxiGen datasets show that PPO-CIS improves detection accuracy by 2.10 percent while increasing processing speed from 42.74 to 384 samples per second compared with static cascade designs. The findings show that adaptive model selection can better shield users from exposure to harmful content while lowering moderation costs. PPO-CIS provides a practical solution for deploying scalable and timely content moderation in fast-moving online environments.

Knowledge-Based SystemsVol. 341
McGill University (CA), Concordia University (CA)
Peace, Justice and strong institutions
Openalex Percentile: Top 34%
Hate Speech and Cyberbullying Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.