LLMDPA: pre-execution privacy auditing of DP-SGD via evidence-constrained semantic analysis

Abstract Privacy auditing has emerged as a critical component for safeguarding data privacy in machine learning models. Existing empirical auditing methods heavily rely on model runtime states or post-deployment interfaces, resulting in substantial time costs and limited source-level localization of leakage causes. In contrast, the theoretical guarantees of provable privacy mechanisms provide a basis for efficient privacy auditing with source-level traceability before model training. However, the validity of such auditing depends on the semantic consistency of the underlying implementation, which remains difficult for current automated methods to verify due to limited privacy-specific reasoning and explicit evidence modeling. To address this gap, we formalize the Pre-execution Privacy Auditing paradigm and instantiate it for DP-SGD through LLMDPA, an automated framework based on evidence-constrained semantic analysis. LLMDPA constructs a structured Evidence Chain to constrain LLM-based adjudication and identify hidden semantic failures in DP-SGD implementations. Evaluations on real-world and controlled benchmarks show that LLMDPA achieves global precision ranging from 98.82 to 100% and global F1-scores ranging from 0.9091 to 0.9677 across four LLM configurations, outperforming conventional static and vanilla LLM baselines. Moreover, LLMDPA audits a repository in approximately 50 s, providing a scalable and efficient complement to scale-sensitive attack evaluations. Overall, LLMDPA provides an interpretable and evidence-traceable instantiation of Pre-execution Privacy Auditing for DP-SGD.