Classifying illicit dark web content through zero-shot prompting: An empirical study with GPT models

This study evaluates the classification performance of four GPT-based models (GPT-4.1, GPT-4.1-mini, GPT-4.1-nano, and o4-mini) under zero-shot prompting conditions on the complete, multilingual CoDA dataset of Dark Web content, comprising 10 illicit activity categories. The models GPT-4.1, GPT-4.1-mini, and o4-mini achieve a weighted F1 score of 0.885, surpassing prior zero-shot baselines on this dataset. Stability analysis using TARa@10 demonstrates high output consistency for GPT-4.1 (0.964) and GPT-4.1-mini (0.970), indicating their reliability for operational use. Multilingual evaluation reveals only a modest English vs. non-English performance gap for GPT-4.1 (0.031), while other models perform comparably across languages. The strongest results appear in Drugs, Gambling, and Porn (F1 > 0.9), whereas lower scores are observed in ambiguous or overlapping categories like Violence (F1 <= 0.76) or Crypto (F1 <= 0.84). A qualitative review of misclassifications suggests that some model predictions align with reasonable semantic interpretations, potentially highlighting annotation inconsistencies. This work establishes a performance baseline for GPT-based models in zero-shot classification of multilingual Dark Web content and underscores the importance of clear category definitions for effective deployment.