AI against abuse: real-time protection and detection of unsafe content from visual and verbal threats on social media

With the rapid growth of video content on platforms such as Instagram and YouTube, the spread of harmful and illegal material by unauthenticated users has become a serious concern, particularly for children, youth, and other vulnerable audiences. This paper presents a real-time, AI-driven multimodal content moderation system implemented to detect and control unsafe content, including nudity, profanity, gore, violence, and hate speech, in both live streams and uploaded videos. The system integrates DeepLabV3 + for semantic segmentation and Faster R-CNN for object detection to analyze visual elements, while Wav2Vec 2.0 and a CNN are used to identify abusive speech and ambient threats such as gunshots and screams. BERT is employed to detect toxic and hate speech in video-related text. A priority-based Directed Acyclic Graph (DAG) scheduler orchestrates these models for efficient execution and load balancing, and Firefly Optimization is used for hyperparameter tuning to enhance accuracy. The implemented multimodal system achieves an overall detection accuracy of 98.33%, and includes a customizable Content Sentinel module that enables real-time actions such as skipping flagged segments. This solution promotes ethical video sharing and offers a scalable approach to content moderation across social media platforms.

Authors

Institutions

Publication Details

Journal
Scientific Reports
Published
2026-11-18
DOI
https://doi.org/10.1038/s41598-026-81051-6
Primary Topic
Hate Speech and Cyberbullying Detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

AI against abuse: real-time protection and detection of unsafe content from visual and verbal threats on social media

D. Nivetha, Manikandan Dhayanithi, M. Swarna
3 links
Scientific Reports
Hate Speech and Cyberbullying Detection
article

AI against abuse: real-time protection and detection of unsafe content from visual and verbal threats on social media

D. Nivetha, Manikandan Dhayanithi, M. Swarna
article en

Abstract

With the rapid growth of video content on platforms such as Instagram and YouTube, the spread of harmful and illegal material by unauthenticated users has become a serious concern, particularly for children, youth, and other vulnerable audiences. This paper presents a real-time, AI-driven multimodal content moderation system implemented to detect and control unsafe content, including nudity, profanity, gore, violence, and hate speech, in both live streams and uploaded videos. The system integrates DeepLabV3 + for semantic segmentation and Faster R-CNN for object detection to analyze visual elements, while Wav2Vec 2.0 and a CNN are used to identify abusive speech and ambient threats such as gunshots and screams. BERT is employed to detect toxic and hate speech in video-related text. A priority-based Directed Acyclic Graph (DAG) scheduler orchestrates these models for efficient execution and load balancing, and Firefly Optimization is used for hyperparameter tuning to enhance accuracy. The implemented multimodal system achieves an overall detection accuracy of 98.33%, and includes a customizable Content Sentinel module that enables real-time actions such as skipping flagged segments. This solution promotes ethical video sharing and offers a scalable approach to content moderation across social media platforms.

Scientific Reports
Vels University (IN)
Peace, justice, and strong institutions
Hate Speech and Cyberbullying Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon