Privacy-Preserving Automated Deep Learning for Secure Inference Service

Automated deep learning (AutoDL) aims to automatically discover optimal architectures of deep neural networks (DNNs) for secure inference without the studies for time-consuming and error-prone manual design. Privacy concerns have increasingly motivated the studies for privacy-preserving AutoDL (PrivAutoDL), where DNN architectures are searched directly on encrypted data without revealing the client's confidential inputs and well-trained DNN architectures. However, existing studies encounter problems in achieving a balance between provable security and efficiency while avoiding significant degradation of model utility. To tackle these problems, we design a privacy-preserving AutoDL scheme, named 2PCAutoDL, utilizing a two-party (two non-colluding cloud servers) computation model. Based on the two-server model, efficient and secure computation protocols are customized layer by layer to protect DNN models associated with client's data. In particular, we reduce the computational overhead of secure DNN: our optimized protocols achieve <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1.34\times \sim 2.05\times$</tex-math></inline-formula> speedup for linear layers and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1.33 \times \sim 45 \times$</tex-math></inline-formula> speedup for non-linear layers, compared to a range of existing secure implementations in the literature. Moreover, our fresh alternative to approximate Softmax avoids the drawbacks of approximating exponential operation and yields slightly higher accuracy under appropriate configurations. The security of 2PCAutoDL is formally analyzed under the semi-honest adversary model. Extensive experiments demonstrate that the searched models from 2PCAutoDL improve the inference accuracy by 0.6% on MNIST and by 0.5% on CIFAR-10 when compared to state-of-the-art (SOTA) PrivAutoDL.