Insights from Hyperparameter Scaling of Online Speech Separation

With the rapid development of deep learning, a large number of models with excellent performance for speech separation tasks have emerged in the literature. Despite their impressive performance, these models usually cannot be used for online speech separation tasks due to high computational and memory costs. In recent years, some approaches tailored for online speech separation have been developed to achieve state-of-the-art performance while keeping the model lightweight. However, it is often unclear how these models can scale with their architecture-specific hyperparameters as they are under-explored in the literature. To fill this gap, this paper investigates the effect of hyper-parameter scaling on the model performance, number of parameters, and computational cost based on two representative online speech separation models, BSRNN and SkiM. Experimental results on the Libri2Mix dataset reveal the relationship between model performance and composite hyperparameter configurations. We also derive the patterns of the number of parameters and computational costs varying with hyperparameters. Based on the results of the analysis, we give suggestions for scaling the two models and summarize two common trends.