Survey on Application of Homomorphic Encryption in Deep Learning

With the widespread application of deep learning in various fields, data privacy and security issues have become increasingly important. Homomorphic encryption, a technique that allows computations to be performed directly on encrypted data, offers a potential solution to these problems. This paper surveys methods that combine deep learning with homomorphic encryption, exploring how to effectively apply deep learning models in encrypted environments. Firstly, the basics of homomorphic encryption are introduced, covering its basic principles, different classifications (including partially homomorphic encryption, somewhat homomorphic encryption and fully homomorphic encryption), and the development history of fully homomorphic encryption. Key models in deep learning, such as convolutional neural network and Transformer, are then detailed. The steps of combining homomorphic encryption with deep learning and how to adapt various layers of deep learning (e.g., convolutional layers, attention layer and activation function layer) to the homomorphic encryption environments are discussed. Subsequently, existing methods that integrate convolutional neural network and Transformer with homomorphic encryption are focused on. Specific implementation schemes for performing deep learning computations on encrypted data and performance optimization strategies employed to enhance efficiency and accuracy are discussed. The advantages and limitations of each method are summarized. Finally, current research progress is summarized, and an outlook on future research directions is provided.