CABY26: Statistics and Analysis of a Cleaned, Re-labeled Cyberbullying Corpus with Agreement-Based Bystander Roles

This paper introduces CABY26, a fully re-labeled version of the CYBY24 cyberbullying corpus. CABY26 replaces CYBY24's four-level bullying severity scale and biased bystander labels ("instigator", "defender") with a clear binary Aggression / No Aggression judgment and an agreement-based bystander role scheme (agree, disagree, impartial, other), following the remediation framework proposed in the companion guidelines paper. Three annotators labeled the full corpus using a chronological three-stage workflow: they judge the main post alone, then assign a role to each reply, then judge the whole thread. Before analysis, we removed 8 threads with pornographic main posts and 234 replies that were exact duplicates, leaving 2,781 threads and 10,241 replies. Inter-annotator agreement (Fleiss' kappa) falls in the fair range for all three label stages (0.33 to 0.39), bystander stance is linked to thread-level aggression (Cramér's V = 0.144, p < 0.001), and the panel changes its verdict from the isolated main post to the full thread in about 13% of threads, confirming the concept of Change of Mind (CoM) that CYBY24 showcased. We also report the practical challenges the annotator panel faced while labeling, especially the lack of image and video context and the difficulty of judging exactly who a reply's aggression is aimed at, and turn these into concrete recommendations for future annotation projects. We then test whether the relabeling helps models. On a held-out test set of 418 threads and 1,523 replies, fine-tuned BERT and RoBERTa and three prompted Large Language Models (LLMs) all score higher on CABY26 bystander role labels than on CYBY24 labels for the same replies: macro-F1 (0-100) rises from 33.6-43.0 to 46.1-53.0. We release a thread-level 70:15:15 split and benchmark results with the corpus.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24237810
Primary Topic
Hate Speech and Cyberbullying Detection
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

CABY26: Statistics and Analysis of a Cleaned, Re-labeled Cyberbullying Corpus with Agreement-Based Bystander Roles

Keng Hoon Gan, Syaheerah Lebai Lutfi, Ramesh Perumal, Hazem A. M. Nabil
Zenodo (CERN European Organization for Nuclear Research)
Hate Speech and Cyberbullying Detection
preprint

CABY26: Statistics and Analysis of a Cleaned, Re-labeled Cyberbullying Corpus with Agreement-Based Bystander Roles

Keng Hoon Gan, Syaheerah Lebai Lutfi, Ramesh Perumal, Hazem A. M. Nabil
preprint en

Abstract

This paper introduces CABY26, a fully re-labeled version of the CYBY24 cyberbullying corpus. CABY26 replaces CYBY24's four-level bullying severity scale and biased bystander labels ("instigator", "defender") with a clear binary Aggression / No Aggression judgment and an agreement-based bystander role scheme (agree, disagree, impartial, other), following the remediation framework proposed in the companion guidelines paper. Three annotators labeled the full corpus using a chronological three-stage workflow: they judge the main post alone, then assign a role to each reply, then judge the whole thread. Before analysis, we removed 8 threads with pornographic main posts and 234 replies that were exact duplicates, leaving 2,781 threads and 10,241 replies. Inter-annotator agreement (Fleiss' kappa) falls in the fair range for all three label stages (0.33 to 0.39), bystander stance is linked to thread-level aggression (Cramér's V = 0.144, p < 0.001), and the panel changes its verdict from the isolated main post to the full thread in about 13% of threads, confirming the concept of Change of Mind (CoM) that CYBY24 showcased. We also report the practical challenges the annotator panel faced while labeling, especially the lack of image and video context and the difficulty of judging exactly who a reply's aggression is aimed at, and turn these into concrete recommendations for future annotation projects. We then test whether the relabeling helps models. On a held-out test set of 418 threads and 1,523 replies, fine-tuned BERT and RoBERTa and three prompted Large Language Models (LLMs) all score higher on CABY26 bystander role labels than on CYBY24 labels for the same replies: macro-F1 (0-100) rises from 33.6-43.0 to 46.1-53.0. We release a thread-level 70:15:15 split and benchmark results with the corpus.

Zenodo (CERN European Organization for Nuclear Research)
Universiti Sains Malaysia (MY), Intel (Malaysia) (MY), Sultan Qaboos University (OM)
Hate Speech and Cyberbullying Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon