Projection Valued-based Quantum Machine Learning Adapting to Differential Privacy Algorithm for Word-level Lipreading

Deep neural network (DNN)-based lipreading models have achieved excellent recognition accuracy but are currently facing challenges related to user privacy. To address this, we propose a novel hybrid quantum-classical neural network (HQCNN) for lipreading that balances superior performance with enhanced privacy protection. The HQCNN-based lipreading model features an innovative variational quantum circuit (VQC) back-end, which transforms the output of the DNN front-end into quantum representations and predicts the posterior probability of each word. Furthermore, we introduce projection-valued encoding (PVE) and projection-valued measurement (PVM), enabling the VQC to handle inputs and outputs of dimensions that scale exponentially with the number of qubits, thereby substantially increasing its expressive power. Additionally, we explore the privacy-preserving properties of the HQCNN-based lipreading model by integrating differentially private stochastic gradient descent (DP-SGD). Experiments conducted on the LRW dataset demonstrate the model’s exceptional recognition accuracy and privacy-preserving capabilities.