AI-Powered Decision Support in Privacy Laws: Benchmarking LLMs on Legal Violation Detection

The integration of Large Language Models (LLMs) into legal practice creates a conflict between performance and data privacy. While proprietary models excel at legal analysis, their cloud-based nature risks breaching regulations, such as the GDPR, by transmitting sensitive data. This study addresses this tension by examining whether locally hosted, open-source LLMs can serve as a viable, privacy-preserving alternative for judicial support. We benchmark five models on classifying GDPR enforcement cases, using chain-of-thought prompting to evaluate accuracy, size, and speed trade-offs. Our findings reveal that while cloud-based models achieve the highest parent-article accuracy (58\%), open-source models offer competitive performance (45\%) while running entirely on-premises. This demonstrates that local models are a potent tool for legal analysis when confidentiality is paramount. \footnote{The code, data, and experimental results will be added to the authors' GitHub repository upon publication.} We conclude by proposing knowledge distillation as a path to creating specialized and secure models, thereby providing a benchmark for developing transparent and auditable AI tools for judicial decision-making.