Private and Efficient Learning With Hyperdimensional Computing

Machine learning algorithms, especially deep neural networks (DNNs), are computationally expensive and vulnerable to privacy breaches. Hyperdimensional Computing (HDC) is a brain-inspired machine learning paradigm which offers a potential alternative for efficient and private learning. HDC encodes dense, sensory inputs into sparse, high-dimensional vectors of 10,000 bits or more, and learns using well-defined and lightweight operations on these vectors. However, naïve HDC is vulnerable to privacy attacks as the encoding process can be reversely determined. In this paper, we systematically study the fundamental privacy challenges of HDC due to reversibility and propose an end-to-end private training and inference framework for HDC with efficient hardware implementation. We first show that HDC is not naturally private as the original data and encoding parameters can be reconstructed from the encoded vector, regardless of the encoding methods applied. We term this attack as the feature extraction attack. We then propose a locally sparse encoding method to obfuscate the encoded information for protecting inference privacy, leading to computational and communication savings. We further propose Privé-HDnn which uses a hybrid architecture of Convolutional Neural Network (CNN) as feature extractors and private-preserved HDC classifier for complex tasks such as image classification. Thanks to the noise tolerance of HDC, Privé-HDnn enables private single- and multi-pass training of HDC by differential privacy-guaranteed noise injection. Results show that our new encoding method defends against decoding attacks with an RMSE deviation of 0.42, while also speeding up encoding by 2.6–5.3<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> (19–79<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\times$</tex-math></inline-formula>) compared to the FPGA (CPU) baseline. For privacy-preserved training, Privé-HDnn achieves comparable or up to 5.8% better accuracy than DNNs with up to 1231<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> faster training time on GPU as compared to the state-of-the-art NNs.