Shot Noise Optimal Receiver Filters for Coherent and Non‐Coherent Fibre Optic Communications

ABSTRACT In this paper, we investigate optimal receiver filter design with respect to shot noise in both non‐coherent and coherent fibre optic communication systems. We derive analytical expressions for the moment‐generating function of filter output and the signal‐to‐shot noise power ratio (SSNR) for intensity modulation direct detection (IM/DD) and phase diversity homodyne (PDH) optical fibre communication systems, with arbitrary waveform and receiver filter considerations in the presence of shot noise. Closed‐form formulas are provided for the receiver filter structures that maximise the SSNR. The performance of SSNR is evaluated using return‐to‐zero (RZ) coding with rectangular and Gaussian pulse shapes across various receiver filter structures, including the ideal integrator, the RC model, and the matched filter. Our results indicate that the integrator, or its variations, is either optimal or near‐optimal for diverse IM/DD systems. However, matched filtering to the received pulse power becomes superior when dark current or d.c. bias power is prominent. For coherent PDH systems, matched filtering to the complex received pulse amplitude approaches near‐optimal performance provided the local oscillator (LO) power significantly exceeds the received signal power. When the LO power is much greater than the signal power, such systems can achieve double the SSNR of IM/DD systems, equivalent to twice the number of converted photoelectrons per symbol, thus nearing the quantum‐limit performance considering the photodetector quantum efficiency.