Dual-Role Evaluation of Large Language Models in Cybersecurity: A Unified Framework for Offensive Generation and Defensive Intelligence

The rapid advancement of large language models (LLMs) is reshaping cybersecurity by strengthening defensive capabilities while simultaneously enabling increasingly sophisticated offensive threats. Despite this dual-use nature, existing evaluation methodologies remain fragmented and commonly assess LLMs in isolation as either offensive or defensive systems. This work presents a unified dual-role evaluation framework that characterizes LLM behavior within a shared cybersecurity context through a multi-dimensional taxonomy encompassing role, capability, intent, and contextual use. The framework is empirically evaluated under a strictly single-turn setting (T = 1), in which each offensive input is independently assessed through one corresponding defensive response. The evaluation covers phishing detection, prompt injection, log analysis, malware reasoning, vulnerability exploitation, and incident response using malicious, benign, and ambiguous scenarios. Experimental results demonstrate strong overall detection performance, low error rates, and effective handling of dual-use and borderline cases through ambiguity-aware classification. The findings further reveal the difficulty of distinguishing malicious intent from structurally similar benign inputs, underscoring the importance of context-sensitive evaluation and uncertainty-aware decision making in cybersecurity applications. Overall, the study establishes a controlled and reproducible single-turn baseline for evaluating offensive inputs and defensive LLM responses across diverse cybersecurity tasks.

Authors

Institutions

Publication Details

Journal
Journal of Engineering Research and Sciences
Published
2026-11-21
DOI
https://doi.org/10.55708/js0511007
Primary Topic
Adversarial Robustness in Machine Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Dual-Role Evaluation of Large Language Models in Cybersecurity: A Unified Framework for Offensive Generation and Defensive Intelligence

Vaibhavi Tiwari, Ramy Othman, Ola Suaifan
4 links
Journal of Engineering Research and Sciences
Adversarial Robustness in Machine Learning
article

Dual-Role Evaluation of Large Language Models in Cybersecurity: A Unified Framework for Offensive Generation and Defensive Intelligence

Vaibhavi Tiwari, Ramy Othman, Ola Suaifan
article en

Abstract

The rapid advancement of large language models (LLMs) is reshaping cybersecurity by strengthening defensive capabilities while simultaneously enabling increasingly sophisticated offensive threats. Despite this dual-use nature, existing evaluation methodologies remain fragmented and commonly assess LLMs in isolation as either offensive or defensive systems. This work presents a unified dual-role evaluation framework that characterizes LLM behavior within a shared cybersecurity context through a multi-dimensional taxonomy encompassing role, capability, intent, and contextual use. The framework is empirically evaluated under a strictly single-turn setting (T = 1), in which each offensive input is independently assessed through one corresponding defensive response. The evaluation covers phishing detection, prompt injection, log analysis, malware reasoning, vulnerability exploitation, and incident response using malicious, benign, and ambiguous scenarios. Experimental results demonstrate strong overall detection performance, low error rates, and effective handling of dual-use and borderline cases through ambiguity-aware classification. The findings further reveal the difficulty of distinguishing malicious intent from structurally similar benign inputs, underscoring the importance of context-sensitive evaluation and uncertainty-aware decision making in cybersecurity applications. Overall, the study establishes a controlled and reproducible single-turn baseline for evaluating offensive inputs and defensive LLM responses across diverse cybersecurity tasks.

Journal of Engineering Research and SciencesVol. 5(11)
Montclair State University (US)
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Dual-Role Evaluation of Large Language Models in Cybersecurity: A Unified Framework for Offensive Generation and Defensive Intelligence — Vaibhavi Tiwari, Ramy Othman, et al. · Journal of Engineering Research and Sciences (2026) | TGRS Research Map | TGRS