Inverse‐Designed on‐Chip Terahertz Three‐Channel Mode and Wavelength Division Demultiplexer

High‐performance multimode/multiwavelength (de)multiplexer is one of the most pivotal photonic devices for advanced on‐chip interconnect systems. Traditional on‐chip photonic (de)multiplexing requires large device footprint for maintaining high efficiency, large operation bandwidth, and small insertion losses (IL). Here, a hybrid inverse design method is therefore proposed to combine genetic algorithms (GA) and topology optimization (TO) for developing an ultracompact (lateral size < 2λ) terahertz (THz) mode‐/wavelength‐division demultiplexer (MDM‐WDM). The method leverages the global search capability of GA in continuous parameter spaces and the local TO strategy driven by the adjoint method, effectively improving design convergence efficiency and global performance robustness. Results demonstrate that the device simultaneously achieves stable three‐channel MDM and WDM with IL of less than 3 dB and inter‐channel crosstalk can be −22 dB. The output achieves spatial separation of the orthogonal TE 10 , TE 20 , and TE 30 modes that correspond to three central wavelengths λ 1 = 690 μm, λ 2 = 700 μm, and λ 3 = 710 μm, respectively, verifying the device's precise control over target modes and wavelengths. This work provides an efficient optimization approach for developing broadband, multichannel, and highly integrated THz multiplexing devices, offering new pathways for constructing next‐generation integrated photonic interconnects and signal processing systems.