Revisiting Text-Based CAPTCHAs: A Large-Scale Security and Usability Analysis Against CNN-Based Solvers

Text-based CAPTCHAs remain a widely deployed mechanism for mitigating automated attacks across web platforms. However, the increasing effectiveness of convolutional neural networks (CNNs) and advanced computer vision models poses significant challenges to their reliability as a security measure. This study presents a comprehensive forensic and security-oriented analysis of text-based CAPTCHA systems, focusing on how individual and combined visual distortion features affect human usability and machine solvability. A real-world dataset comprising 45,166 CAPTCHA samples was generated under controlled conditions, integrating diverse anti-recognition, anti-segmentation, and anti-classification features. Recognition performance was systematically evaluated using both a CNN-based solver and actual human interaction data collected through an online exam platform. Results reveal that while traditional features such as warping and distortion still degrade machine accuracy to some extent, newer features like the hollow scheme and multi-layer structures offer better resistance against CNN-based attacks while maintaining human readability. Correlation and SHAP-based analyses were employed to quantify feature influence and identify configurations that optimize human–machine separability. This work contributes a publicly available dataset and a feature-impact framework, enabling deeper investigations into adversarial robustness, CAPTCHA resistance modeling, and security-aware human interaction systems. The findings underscore the need for adaptive CAPTCHA mechanisms that are both human-centric and resilient against evolving AI-based attacks.