Uncertainty-Based Remixing for Unsupervised Domain Adaptation in Deep Speech Enhancement

Recent work has shown the effectiveness of remixing-based unsupervised domain adaption algorithms, where a student model is fine-tuned on self-labeled noisy-clean speech data synthesized by remixing speech and noise predictions from the teacher model. However, the optimization of the student model may be hindered by learning from fundamentally erroneous pseudo-targets created by the teacher model. To address this limitation, we augment the teacher model with an uncertainty estimation task and propose an uncertainty-based remixing method that allows the student model to learn from the teacher model's high-quality speech estimates and effectively suppress noise. Experiments demonstrate improved robustness against data mismatches between training and testing conditions, especially for challenging inputs with low signal-to-noise ratios. Moreover, by adjusting the uncertainty threshold to categorize the teacher's estimates for unlabeled noisy samples as reliable or unreliable, the proposed uncertainty-based remixing process allows for a controllable trade-off between noise suppression and speech preservation, enabling the model to be adapted to diverse application needs.