Structural Immunity and Silent Compromise: Prompt Injection Against LLM-Based and Traditional ML Text Classifiers

Large language models are increasingly deployed as drop-in replacements for supervised text classifiers in content moderation, triage and filtering pipelines. The prompt injection literature has concentrated on agentic and retrieval-augmented systems, where the vulnerability is well documented. The classifier-replacement setting has received far less attention, and it differs in one respect that matters: the system being replaced is still available as an experimental control. This paper uses that control. Three TF-IDF classifiers (Naive Bayes, SVM, Logistic Regression) and three LLM classifiers (GPT-4, Llama 3 8B, Mistral 7B) are evaluated on the SMS Spam Collection dataset under four prompt injection strategies, using all 149 spam messages in the test split plus 200 ham messages, with repeats at stochastic temperature, for 15,705 model queries. Two results are reported. First, and consistent with prior work on capability and susceptibility, every instruction-following model failed on the majority of injected spam under a direct instruction override: Llama 3 on 133 of 133 previously-detected spam messages (100%, 95% Wilson CI 97.2 to 100), Mistral on 143 of 144 (99.3%, CI 96.2 to 99.9), and GPT-4 on 100 of 143 (69.9%, CI 62.0 to 76.8). The TF-IDF classifiers were unaffected, not through robustness but because a bag-of-words representation cannot encode an instruction. GPT-4 recorded the highest clean accuracy of any model tested. Second, and less expected, output-format validation defends the weakest model and not the others. Of 15,705 responses, only 8 were genuinely uninterpretable. GPT-4 and Llama 3 returned a bare single-token label on every one of their 12,215 queries, so a strict format validator would reject none of their successful evasions. Mistral narrated its own compliance on 141 of 145 successful role-play evasions, all of which such a validator would reject. Narration appears to be an artefact of weaker instruction-following rather than a signature of attack failure, which implies that artefact-based detection measured on small models will overstate its protection on larger ones. This is offered as a hypothesis supported by a three-model contrast, not as an established scaling law. Decoding temperature was measured rather than assumed. Local models run at 0.0 and 0.8 differed by at most 3.5 percentage points on any strategy. This version supersedes Version 1, which is retained for reference. Version 1 reported that GPT-4 resisted nearly all injection attacks on the basis of ten spam messages; at full sample that finding is wrong and is retracted. Section 8 documents every change.