High-Performance Shape-Optimized Lithium Niobate-on-Insulator Mode (De)multiplexer

Mode-division multiplexing (MDM) technology significantly increases transmission capacity, with mode (de)multiplexers serving as key components. However, designing lithium niobate-on-insulator (LNOI) mode (de)multiplexers is challenging due to potential mode hybridization. In this work, we propose an ultra-compact and high-performance mode (de)multiplexer based on inverse-design shape optimization for LNOI platform. By accounting for crystal anisotropy, sidewall angles, rib geometry, and minimum feature sizes, we optimize the device structure to achieve a compact device with low insertion loss (IL) and crosstalk (CT). The experimental results for the fabricated four-channel mode (de)multiplexer show IL of approximately 0.37 dB and maintains CT below −15.7 dB over a wide wavelength range of 1500–1600 nm. These results highlight the potential of LNOI in multimode photonics and reveal its suitability for high-density, low-loss photonic devices.