Towards Efficient Privacy-Preserving Deep Learning: HE-Friendly Structures, Flexible Pruning, HE-Efficient Architectures, and Secure Transformer Token Dro

Deep learning (DL) has become a powerful tool for solving complex problems, but developing DL models typically requires vast datasets, high computational resources, and expert knowledge—barriers that limit accessibility. Machine Learning as a Service (MLaaS) addresses this challenge by allowing resource-rich providers to deliver pre-trained DL models as services. However, privacy concerns arise: clients hesitate to share sensitive data, while providers protect their proprietary models. To address this, privacy-preserving MLaaS integrates cryptographic techniques into DL computations, as seen in frameworks like Cryptonets, SecureML, GAZELLE, CrypTFlow2, Cheetah, and BOLT. Among them, Homomorphic Encryption (HE) enables computation on encrypted data but remains computationally expensive. This dissertation presents four novel contributions to enhance HE-based deep learning efficiency: First, Hunter introduces structured pruning using HE-friendly features—internal/external structure and weight diagonal—to reduce HE operations like Perm, Mult, and Add while maintaining accuracy. For instance, on VGG-16 (ImageNet), Hunter cuts Perm to 2% and Mult/Add to 14%. Second, MOSAIC addresses pruning structure inflation by assembling compact, HE-friendly units via intelligent channel transformation, reducing costs by 21.14% and 29.49% on VGG-16 under GAZELLE and CrypTFlow2, respectively. StriaNet designs HE-efficient networks from scratch using StriaBlock, which includes ExPerm-Free Convolution and Cross Kernel modules. It minimizes Perm operations and adapts dimensions to HE characteristics, achieving up to 9.78× speedup on ImageNet and 9.24× on CIFAR-10 over ResNet. Lastly, SecDTD enables efficient Transformer inference via early-stage dynamic token dropping. By integrating Pre-Softmax Scoring (MCN) and Fast Oblivious Median Selection (OMSel), it reduces overhead and achieves up to 16.9× speedup. On BERT-base, it delivers up to 4.47× acceleration on GLUE datasets under BOLT and BumbleBee. Together, these contributions significantly advance the efficiency and practicality of privacy-preserving deep learning, providing a robust foundation for scalable and efficient privacy-preserving MLaaS frameworks.