Specialized and effective neural architecture search for differentially private stochastic gradient descent

Differential privacy (DP) provides rigorous protection against information leakage in deep learning, yet it can substantially compromise model utility by distorting optimization through gradient clipping and noise injection. While most prior work has focused on improving private optimizers, architecture-level studies under DP remain largely manual or fragmented, lacking a systematic framework for designing architectures optimized for private training. Here we introduce SENAS (Specialized and Effective Neural Architecture Search for Differentially Private Stochastic Gradient Descent), which uses a public proxy subset to identify architectures tailored to DPSGD without accessing sensitive data during search. This separation avoids repeated private architecture evaluations and preserves a clear privacy accounting boundary for final DPSGD training. SENAS favors structurally efficient architectures that reduce redundant transformations and improve training stability under clipped and noised gradients. Across four benchmark datasets and privacy budgets ε ranging from 1 to 3, SENAS achieves competitive accuracy, with paired tests indicating statistically significant improvements under several experimental settings. On CIFAR-100 and Imagenette, SENAS further achieves higher accuracy with fewer parameters and lower training cost, demonstrating its scalability to more challenging private learning settings. These findings indicate that architecture design can provide a practical direction for improving the trade-off between privacy and utility in differentially private deep learning.