Quantum noise from vacuum field injection in optical cavities with diffraction-related loss

The space-based gravitational wave detector DECIGO is designed to observe primordial gravitational waves with 1000 km Fabry-Perot cavities. Its sensitivity is limited by quantum noise, and although squeezing can suppress it, its effectiveness is reduced by diffraction-related loss, which leads to the injection of vacuum fields into the interferometer. This paper presents a rigorous treatment of quantum field propagation in the presence of diffraction and higher-order mode losses, deriving input-output relations and modeling their impact via an optomechanical block diagram. The analysis shows that diffraction-induced vacuum fields slightly increase radiation pressure noise, while shot noise remains unaffected. Nevertheless, cavity detuning with homodyne detection yields a dip in the noise spectrum. By accurately capturing these effects, this framework enables a detailed study of sensitivity improvements made by either just detuning the main cavity while implementing homodyne detection, or by combining this with optical-spring quantum locking using auxiliary cavities, laying a firm foundation for enhancing DECIGO's capability to detect primordial gravitational waves.