Vib2audio: Robust Sound Recovery With Micro Vibration Sensing Via mmWave Radar

The popularity of smartphones presents a potential risk to personal privacy, particularly through covert eavesdropping by external non-acoustic sensors (e.g., high-speed camera, Radio Frequency (RF) sensor), where RF-based voice perception technology shows great potential due to its insensitivity to the environment and its ability to traverse occlusions. In this paper, we introduce Vib2audio, a high-quality audio recovery system that uses RF signals to recover audio from the tiny vibration of sound source. We discover that the extracted phases loss high-frequency bands of vibration. Therefore, we investigate the frequency detectable range of mmWave radar used in our scenario and develop an improved spectrum reconstruction network based on the pix2pix structure to accurately recover the time-frequency spectrum of audio from the limited frequency band of the vibration signal. In addition, we synthesize the audio waveform of voice from the time-frequency spectrum using a Griffin-Lim vocoder. Our extensive experiments indicate that Vib2audio can robustly recover tiny sound source vibration to recognizable high-quality human voice audio from tiny vibration. In our overall objective and subjective performance evaluation, the reconstructed audio by Vib2audio had a MAE of 3.6% in the NISQA and achieved a subjective identifiability assessment score of 3.48. Furthermore, the quantitative evaluation results show that Vib2audio is effective and robust to background noise and multi-source interference.