A Framework for Evaluating Robustness of Cybersecurity Named Entity Recognition: Clean Baseline and Controlled Perturbation Design — Preliminary Report

Cybersecurity named entity recognition (NER) systems are commonly evaluated on relatively clean text, while real-world security communication may contain multilingual content, code-switching, and deliberate or accidental text obfuscation. This preliminary report presents a reproducible framework for evaluating cybersecurity NER under these conditions. The study uses the AnnoCTR cybersecurity entity dataset and a DistilBERT token-classification model. A controlled evaluation protocol is defined for multilingual context changes, Arabic-English code-switching, surface-form obfuscation, and normalization. On an initial held-out subset of 200 examples, the clean English baseline produced token-level precision of 0.7369, recall of 0.8584, and F1 of 0.7931. These values are preliminary and are reported from one initial configuration; they have not yet been supplemented by multi-seed variance or span-level evaluation. Perturbation results are intentionally not claimed in this version. The purpose of this release is to document the experimental framework and establish a reproducible baseline before the robustness experiments are completed.