Arti-Invar: A Pre-trained Model for Enhancing Acoustic-to-Articulatory Inversion Performance

Acoustic-to-Articulatory Inversion (AAI) aims to reconstruct the trajectories of articulators from speech signals. The training of an inversion model involves learning information about articulatory invariants and speaker idiosyncrasies. While the former needs only a limited amount of data, the latter requires far more data to learn and is thus a main contributor of low performances as the AAI task suffers a scarcity of articulatory data. Regarding this problem, we proposed transferring the task of learning linguistic invariants to a model pre-trained on a single corpus and combining it with a normal inversion model to allow the latter to better exploit given data for speaker idiosyncrasies. The pre-trained model, referred to as Arti-Invar model, consists of a Content Encoder Module and an Inversion Module and was trained using a PCC-based loss function. We tested several manners of applying it to AAI tasks performed on other corpora and found that taking its output as an additional input features could bring slight improvements on reconstruction scores (0.07 mm). Further experiments on a Chinese EMA corpus proved its generalizability.