Novel learning-based multi-modal methods for human voice apparatus imaging

Human speech production is a complex process that involves generating vibrations at vocal folds, and modulating breath through vocal tract shaping to produce meaningful sounds primarily for communication and social interaction. Vocal apparatus refers to the human vocal tract anatomy responsible for sound generation, and consists of several structures such as larynx, glottis, pharynx, soft/hard palate, tongue, lips, and nose. Disorders in any of these components can compromise one’s ability to produce effective language and communication. Visualizing the complex movements of vocal apparatus is crucial to advance our understanding of speech production mechanism, diagnose speech disorders, optimize speech therapy, aid in planning surgical procedures on speech organs, and improve various technologies related to speech synthesis and recognition. However, current imaging modalities possess several technical limitations. X-ray Computed Tomography (CT) can clearly detect air-tissue boundaries, and bony structures, but it faces limitations due to radiation exposure. Ultrasound can safely image at high temporal resolution, but cannot capture deeper structures (e.g., glottis), and has poor soft-tissue contrast. Electromagnetic articulography (EMA) and Electropalatography (EPG) work as kinematic tracking systems that measure movement and position of articulators in real time, but it is invasive. Optical endoscopy uses a flexible optical fiber with a camera to visualize nose and larynx, but it invasively deforms the anatomy. Magnetic Resonance Imaging (MRI) is emerging as a powerful modality for dynamic vocal apparatus imaging during speech due to its excellent soft-tissue contrast, non-ionizing radiation, and capability to image along any arbitrary plane orientations, but challenges remain for its widespread adaption. For example, due to device physics, MR signals are sensitive to off-resonance at air and tissue boundaries, are susceptible to motion blurring and lose spatial and temporal features while trying to model fast arbitrary speech motion. There is, therefore, a critical need for an imaging modality that can safely and comprehensively visualize vocal tract shaping during speech production with high spatial and temporal fidelity. This thesis develops novel vocal apparatus MRI techniques for speech and voice production tasks at 3 Tesla. Specifically, this thesis has four components related to imaging various aspects of the vocal apparatus: (1) Development and evaluation of a novel motion-robust, analysis manifold learning-based MRI reconstruction method for multi-slice 2D dynamic speech imaging with improved spatio-temporal resolution at 3 Tesla; (2) Development of a motion-robust, variational manifold learning-based MRI reconstruction method for time-aligned 2D multi-slice (aka pseudo-3D) dynamic speech imaging which enables extraction of vocal tract functions that quantify 3D kinematics of vocal tract shaping; (3) Feasibility study and development of novel motion-robust, constrained reconstruction based dynamic laryngeal MRI reconstruction method to visualization of gross changes in glottic configuration during speech and breathing; (4) Development of a 3D framework to combine bony structures from ultra-low dose CT and soft-tissue structures from MRI to create a 3D high resolution hybrid CT-MRI model of vocal tract. The effectiveness of our methods was validated by several experimental data, blind image quality analysis by experts, and in-vivo experiments in a range of speech science applications.