A two-stage carrier phase recovery scheme based on OTD–MML for PM PS-16QAM coherent optical communication systems

As fiber optic communication systems evolve toward high-speed and long-distance transmission, carrier phase noise has become one of the key factors limiting system performance improvement. Although traditional approaches such as blind phase search perform well under certain conditions, their phase estimation accuracy significantly degrades in probability shaping (PS) systems, particularly under conditions of high linewidth (LW) and strong noise, making them unsuitable for the practical demands of high-order modulation systems. To address this, this paper proposes a two-stage carrier phase recovery scheme based on optimal threshold division and minimum metric decision combined with maximum likelihood phase compensation (OTD–MML), suitable for polarization-multiplexed probability shaping 16QAM systems (PM PS-16QAM). To validate the proposed method, a 40 GBaud PM PS-16QAM transmission system was constructed on the VPItransmissionMaker10.1 (VPI) simulation platform. Simulation results demonstrate that in a system with an entropy of 3.31 bits/symbol, the OTD–MML scheme achieves a 0.26 dB optical signal-to-noise ratio (OSNR) gain compared to the blind phase search (BPS) algorithm. Compared to the principal component phase estimation (PCPE) scheme and the traditional circle division method (TCD), it achieves gains of approximately 0.35 dB and 0.38 dB, respectively. When the OSNR is 13 dB, the EVM values for BPS, PCPE, TCD, and OTD–MML are 16.82%, 14.33%, 14.59%, and 13.70%, respectively. Furthermore, this approach demonstrates superior system tolerance with a maximum LW tolerance of 3 MHz, stronger phase noise suppression capability, more stable convergence characteristics, and higher tolerance to amplifier self-radiation noise. This research provides a high-precision, robust carrier phase recovery solution for high-speed probabilistic phase-modulated coherent optical communication systems, holding significant theoretical value and engineering significance for advancing next-generation coherent optical communication technology.