Maximizing the Capabilities of Tiny Speech Foundation Models in a Privacy Preserving Manner

Voice assistive technologies have given rise to extensive privacy and security concerns. In this paper we investigate whether robust automatic speech recognition (ASR) can be done in a privacy preserving manner on resource constrained devices. For inference, speech recognition systems rely on cloud service providers that host state of the art (SOTA) foundation models (FM) demanding high memory usage and computation complexity. Considering the infeasibility to run these SOTA FM on-device, users are forced to offload their sensitive content to cloud for better inference compromising privacy. Sanitization can be done on speech content at the edge keeping in mind that it has to avoid compromising the ASR accuracy. Our solution is based on the observation that although tiny on-device FM cannot perform robust ASR, they are powerful enough to mask the sensitive segments in speech. Given this observation, we introduce SpeechGuard - a privacy-preserving, hybrid edge-cloud ASR inference engine that is compute inexpensive. We present a novel times-tamp based entity filtering mechanism at the edge and a corresponding confidence score based recovery approach. Our results denote that SpeechGuard offers ASR accuracy and privacy guarantee comparable to existing SOTA FM, while achieving significant speedup and reduction in memory usage.