Hiding a Voice From Speaker Encoders: Adversarial Protection Trained Over Measured Room Channels

Abstract Adversarial perturbations can stop a speaker encoder from extracting a talker’s identity. The tools are standard—Projected Gradient Descent (PGD), Expectation over Transformation (EOT), and room impulse responses—but the closest protection methods optimise without any room response, so it is open whether the channel model used in optimisation matters. We contribute an experimental protocol and its findings. We measured four acoustic channels by exponential sine sweep with consumer devices, trained the protection over three, and tested it on the fourth. We evaluated it using an independently trained encoder and a threshold set under the evaluation conditions. At ε = 0.004 (SNR 23.9 dB, PESQ 2.07, ESTOI 0.86), the independent encoder’s identification rate falls from 98% to 28% (20 speakers × 4 held-out channels, 5 runs). Against simulated channels matched in number, reverberation time, and post-processing, training on measured channels lowers the judge’s similarity by a further 0.045 (95% CI 0.036–0.054) at equal budget and 0.039 at equal SNR. A control shows that about 80% of this advantage comes from a spectral colouring shared by all measured responses, including the held-out test channel, much of it made by our deconvolution. With it added, simulated responses come within 0.007, and a larger simulated set lowers the similarity further than the measured ones. In a feasibility test with one clip re-recorded in four rooms, free of this processing, the protection lowered the judge’s similarity in every room, but less than through the channel models, and the adversary still succeeded in three of the four. In a controlled listening test with 28 listeners, the protected speech was rated 1.2 points lower in quality (4.15 to 2.97 on a five-point scale) and judged the same talker less often (76% to 60%), with no detectable change in the share of words understood.