Scaling Laws Do Not Predict Adversarial Robustness in LLM-Based Security Detection Systems

Large language models (LLMs) are increasingly deployed as automateddefenders in security operations, generating detection rules from attacktelemetry. This paper originally claimed that model size does not predictadversarial detection robustness, based on benchmarking five open-weightmodels (3.8B-14B parameters) with DABS (Dual Adversarial Benchmark Score),a standardized 0-100 metric introduced in this work. Version 1.1 update (2026-09-07): A subsequent audit found that theoriginal five-model result was not reproducible under a corrected,seed-controlled protocol -- see the attached ERRATA.pdf for five specificcorrections, and a replication study. Re-running the original modelsunder seeding did not reproduce the original values (14-33% per-modeldeviation, model ranking scrambled), and expanding to a 13-model grid(1.2B-117B parameters) under the corrected protocol yields a numericallypositive but statistically non-significant trend (R²=0.28, p=0.058). Theoriginal claim is treated as not surviving replication. Full detail,code, and data: https://github.com/0xDanielSec/duel-framework DUEL (Dual Unified Evasion Loop) is an adversarial red-teaming frameworkin which an LLM Attacker generates synthetic Microsoft Sentinel-compatibletelemetry and an LLM Defender generates KQL detection rules across 38MITRE ATT&CK techniques. Framework, benchmark, and dataset available atthe repository above.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-07
DOI
https://doi.org/10.5281/zenodo.20098145
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Scaling Laws Do Not Predict Adversarial Robustness in LLM-Based Security Detection Systems

Daniel Gomes Viana Rocha
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
preprint

Scaling Laws Do Not Predict Adversarial Robustness in LLM-Based Security Detection Systems

Daniel Gomes Viana Rocha
preprint en

Abstract

Large language models (LLMs) are increasingly deployed as automateddefenders in security operations, generating detection rules from attacktelemetry. This paper originally claimed that model size does not predictadversarial detection robustness, based on benchmarking five open-weightmodels (3.8B-14B parameters) with DABS (Dual Adversarial Benchmark Score),a standardized 0-100 metric introduced in this work. Version 1.1 update (2026-09-07): A subsequent audit found that theoriginal five-model result was not reproducible under a corrected,seed-controlled protocol -- see the attached ERRATA.pdf for five specificcorrections, and a replication study. Re-running the original modelsunder seeding did not reproduce the original values (14-33% per-modeldeviation, model ranking scrambled), and expanding to a 13-model grid(1.2B-117B parameters) under the corrected protocol yields a numericallypositive but statistically non-significant trend (R²=0.28, p=0.058). Theoriginal claim is treated as not surviving replication. Full detail,code, and data: https://github.com/0xDanielSec/duel-framework DUEL (Dual Unified Evasion Loop) is an adversarial red-teaming frameworkin which an LLM Attacker generates synthetic Microsoft Sentinel-compatibletelemetry and an LLM Defender generates KQL detection rules across 38MITRE ATT&CK techniques. Framework, benchmark, and dataset available atthe repository above.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.