HEDEL: Homomorphically Encrypted Distributed Ensemble Learning

Privacy-Preserving Machine and Deep Learning (PP-MDL) leverages Homomorphic Encryption (HE) to enable the inference and training of Machine and Deep Learning (ML and DL) models on encrypted data, addressing the stringent privacy requirements of domains such as healthcare and finance. While PP-MDL has achieved maturity in inference tasks, training on encrypted data remains a significant challenge due to the computational overhead and operational constraints of HE. This paper introduces Homomorphically Encrypted Distributed Ensemble Learning (HEDEL), a distributed training architecture that enables the training of encrypted ensemble models, starting from encrypted base models and encrypted datasets, by combining Transfer Learning (TL) and Multi-Key Homomorphic Encryption (MKHE). HEDEL presents two unique features. First, TL enables HEDEL to dramatically reduce the computational cost of encrypted training by requiring only a few training epochs to achieve high-accuracy models, making it particularly effective in data-scarce scenarios. Second, MKHE provides a fine-grained access control mechanism for datasets and base models. Through collaborative decryption, model providers can not only regulate who accesses their models but also limit how often these models are used for training and inference. This ensures secure, controlled usage while preventing unauthorized or excessive access, even when a third party offers HEDEL. By combining the efficiency of TL with the security and control of MKHE, HEDEL provides a robust and scalable solution for PP-MDL in collaborative environments involving model providers, who share encrypted models to enable TL, and users who seek accurate, privacy-preserving predictions on their private datasets. Experimental results highlight its ability to overcome HE constraints while delivering high accuracy and strong privacy guarantees.