Patient-Aware Acoustic-to-Articulatory Inversion Through Speech SSL Layer Analysis

We propose a novel patient-aware acoustic-to-articulatory inversion (AAI) model by enhancing the framework of a previously introduced speech self-supervised learning (SSL)-based AAI model. The previous method's reliance on representations extracted solely from the final layer of the SSL model resulted in performance variability depending on patient characteristics. In our work, we conduct an analysis of the features extracted from different SSL layers, which leads to improved performance in patient-aware AAI scenarios. Specifically, we modified the process of capturing the characteristics of multi-channel articulatory signals to better accommodate patient-specific scenarios. Our method achieves a Pearson correlation coefficient of 0.874, demonstrating significantly improved performance compared to the baseline score of 0.847.