Trellis Shaping with Hierarchical Modulation for Nonlinearity Mitigation in Photonic THz Systems

This study investigates trellis shaping (TS) for mitigating nonlinear impairments in photonic terahertz (THz) communication systems. A parallel shaping architecture integrating hierarchical modulation (HM) with adjustable power allocation is proposed. The scheme applies TS to structure the data stream hierarchically and employs superposition coding (SC) to generate non-uniformly distributed TS-16QAM signals, enhancing transmission performance. Experimental validation over a 5-m link at 320 GHz demonstrates that the proposed scheme achieves a 1.65 dB improvement in input power sensitivity and a 1.39 dB gain in Q-factor compared to the Maxwell–Boltzmann (MB) distribution. Furthermore, the system provides enhanced power allocation flexibility, expanding the tunable range of the hierarchical power ratio by a factor of 1.37.