SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection

In the era of rapidly evolving large language models (LLMs), state-of-the-art rumor detection systems, particularly those based on Message Propagation Trees (MPTs), which represent a conversation tree with the post as its root and the replies as its descendants, are facing increasing threats from adversarial attacks that leverage LLMs to generate and inject malicious messages.Existing methods are based on the assumption that different nodes exhibit varying degrees of influence on predictions.They define nodes with high predictive influence as important nodes and target them for attacks.If the model treats nodes' predictive influence more uniformly, attackers will find it harder to target high predictive influence nodes.In this paper, we propose Similarizing the predictive Influence of Nodes with Contrastive Learning (SINCon), a defense mechanism that encourages the model to learn graph representations where nodes with varying importance have a more uniform influence on predictions.Extensive experiments on the Twitter and Weibo datasets demonstrate that SINCon not only preserves high classification accuracy on clean data but also significantly enhances resistance against LLM-driven message injection attacks.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.acl-long.617
Primary Topic
Misinformation and Its Impacts
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection

Shu Wu, Xiang Tao, Qiang Liu, Mingqing Zhang et al.
Misinformation and Its Impacts
article

SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection

Shu Wu, Xiang Tao, Qiang Liu, Mingqing Zhang, Wang Liang
article en

Abstract

In the era of rapidly evolving large language models (LLMs), state-of-the-art rumor detection systems, particularly those based on Message Propagation Trees (MPTs), which represent a conversation tree with the post as its root and the replies as its descendants, are facing increasing threats from adversarial attacks that leverage LLMs to generate and inject malicious messages.Existing methods are based on the assumption that different nodes exhibit varying degrees of influence on predictions.They define nodes with high predictive influence as important nodes and target them for attacks.If the model treats nodes' predictive influence more uniformly, attackers will find it harder to target high predictive influence nodes.In this paper, we propose Similarizing the predictive Influence of Nodes with Contrastive Learning (SINCon), a defense mechanism that encourages the model to learn graph representations where nodes with varying importance have a more uniform influence on predictions.Extensive experiments on the Twitter and Weibo datasets demonstrate that SINCon not only preserves high classification accuracy on clean data but also significantly enhances resistance against LLM-driven message injection attacks.

National Natural Science Foundation of China
Openalex Percentile: Top 74%
Misinformation and Its Impacts
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.