Probing gravity beyond general relativity with bispectrum multipoles of cosmological tracers: I. Theoretical Foundations

Abstract The bispectrum, being sensitive to non-Gaussianity and mode coupling of cosmological fields induced by non-linear gravitational evolution, serves as a powerful probe for detecting deviations from general relativity (GR). The signatures of modified gravity in the bispectrum are even more pronounced in redshift space, where anisotropies from peculiar velocities provide unbiased information on higher-order properties of gravity. We investigate the potential of all non-zero angular multipoles $B_l^m$ of redshift space bispectrum across all possible triangle configurations to probe degenerate higher-order scalar tensor (DHOST) theory. We show that the higher-order multipoles of the bispectrum with $l=2,4,6$ are more sensitive to the modifications in gravity than the spherically averaged monopole moment $l=0$ . These multipoles demonstrate remarkable sensitivity to the higher-order growth history, which varies across triangle configurations, with acute triangles generally being the most sensitive to modification in GR. The values of various multipoles exhibit opposite signs in modified gravity compared to those predicted in GR, which serves as a robust indicator of the deviation from GR. We demonstrate that, unlike $l=2$ and 4 multipoles, the $l=6$ multipoles with $m\leq 4$ are not affected by the quadratic bias and second-order tidal bias parameters, emphasising the need to leverage their capabilities in analyses. The $(l=6, m \gt 4)$ multipoles fail to capture the second-order growth, while all $l=8$ multipoles lack any independent information regarding modified gravity in both linear and nonlinear regimes.