Bulkhead: an Optimized Privacy-Preserving Neural Network Inference System Design

Privacy-preserving neural network inference systems represent a promising technological concept, aiming to enable a server to perform neural network predictions without gaining access to clients' private input data. Such systems effectively address the privacy risks associated with machine learning as a service. Although numerous approaches have been proposed in this area, a practical and real-world applicable design remains elusive, primarily due to the substantial computational and communication overhead introduced by the underlying cryptographic protocols. In this work, we present Bulkhead, an optimized privacy-preserving neural network inference system that significantly reduces both computation and communication costs in secure neural network evaluation. Our key contribution lies in the development of a novel methodology that minimizes the number of invocations to secure computation protocols when evaluating non-linear operations, commonly used in neural network models. We evaluate Bulkhead across three different neural network architectures. Experimental results demonstrate that achieves approximately 33% reduction in both computation and communication costs compared to state-of-the-art systems running on CPUs.