Eve Said Yes: AirBone Authentication for Head-Wearable Smart Voice Assistant

Recent advances in speech and language processing have led to the rise of smart voice services like Alexa, Google Home, and Siri. However, these advancements also increase security risks due to sophisticated voice domain attacks. Instead of relying on acoustic clues to detect replayed or synthesized speech, we utilize microphones and motion sensors in head-wearable devices to authorize legitimate users through bone-conducted vibrations, enabling multi-factor authentication (MFA) for spoken voice. Our proposed two-stage authentication system, AirBone, captures air and bone conduction (AirBone) signals and exploits two authentication factors sequentially. The first stage, called temporal consistency scoring (TCS), employs signal processing to verify the recorded AC and BC signals are concurrent and originate from the same vocalization process. Statistical tools are employed to distinguish legitimate attempts against false-triggering or acoustic attacks. The second stage leverages deep learning to verify the user’s unique bone conduction patterns in the vibration domain. Specifically, we enhance the robustness through data augmentation with constant-Q transform and adversarial training, improving the model’s ability to detect impersonation and machine-induced vibrations. Thanks to these designs, AirBone authentication offers enhanced security via MFA with no extra cost of user effort. In addition, our experimental results demonstrate a <inline-formula><tex-math notation="LaTeX">$96.3\%$</tex-math></inline-formula> overall accuracy, robustness against AirBone noise and room impulse responses, and <inline-formula><tex-math notation="LaTeX">$0.3\%$</tex-math></inline-formula> Equal Error Rate (EER) against acoustic and cross-domain attacks.