Optimization of CNN inference for multi-image with fully homomorphic encryption

Abstract While fully homomorphic encryption (FHE) provides strong security guarantees for privacy-preserving machine learning, its practical adoption is hindered by substantial computational overhead where bootstrapping operations dominate 84% of inference latency. Existing single-image approaches further suffer from significant ciphertext slot underutilization in convolutional layers. To address these limitations, we propose a novel multi-image homomorphic convolution framework featuring three key innovations: (1) simultaneous multi-image packing enabling batched convolution at single-image computational cost, (2) complex number encoding utilizing both real and imaginary components to double slot utilization, and (3) sparsity-aware ciphertext merging that dynamically consolidates underutilized slots. Our comprehensive evaluation on CIFAR-10, CIFAR-100 and ImageNet using ResNet architectures demonstrates that processing 16 images concurrently achieves 74% latency reduction compared to single-image baselines while maintaining a 30% speed advantage over multi-image variants of prior work with negligible accuracy degradation (< 0.7% drop). These results establish a new state-of-the-art for efficient and scalable privacy-preserving inference.