Tilt-to-length coupling coefficients estimation and noise subtraction using a marginalization likelihood function

Space-based gravitational wave (GW) detectors aim to measure GWs in the millihertz frequency band. Various noise sources can limit the GW detection sensitivity during the detection process. Tilt-to-length (TTL) noise, arising from the angular and lateral jitter of spacecraft (SC) and movable optical subassembly (MOSA), is one of the primary noise sources remaining after using the time-delay interferometry (TDI) technique to suppress laser frequency noise. In this paper, we derive an analytical expression for the TTL noise power spectral density (PSD) in arbitrary geometric TDI combinations. Moreover, we calculate the signal-to-noise ratio (SNR) of TTL noise to the noise floor (test mass acceleration noise and optical path noise) and find that the SNR exceeds 1 in the frequency band of 2 mHz to 1 Hz. Then, we construct a marginal likelihood function to estimate TTL coupling coefficients from the unknown noise floor PSD and unknown GW signals. This approach uses these estimated coefficients to subtract TTL noise. We use a simulation to show that TTL coupling coefficients estimation error is less than $0.1\text{ }\text{ }\mathrm{mm}/\mathrm{rad}$ for forty-five geometric TDI combinations, and the suppressed TTL noise level is below the noise floor. Finally, a frequency-domain burst is injected to verify the algorithm's robustness. We compare the estimation accuracy of the marginal likelihood estimation with that of the least squares method. Our work shows that the marginal likelihood estimation effectively subtracts TTL noise and demonstrates stronger robustness in the presence of an unexpected frequency-domain burst.