Analytic properties of an orthogonal Fourier–Jacobi Dirichlet series

Abstract We investigate the analytic properties of a Dirichlet series involving the Fourier–Jacobi coefficients of two cusp forms for orthogonal groups of signature $$(2,n+2)$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mo>(</mml:mo> <mml:mn>2</mml:mn> <mml:mo>,</mml:mo> <mml:mi>n</mml:mi> <mml:mo>+</mml:mo> <mml:mn>2</mml:mn> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> . Using an orthogonal Eisenstein series of Klingen type, we obtain an integral representation for this Dirichlet series. In the case when the corresponding lattice has only one 1-dimensional cusp, we rewrite this Eisenstein series in the form of an Epstein zeta function. If additionally $$4 \mid n$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>4</mml:mn> <mml:mo>∣</mml:mo> <mml:mi>n</mml:mi> </mml:mrow> </mml:math> , we deduce a theta correspondence between this Eisenstein series and a Siegel Eisenstein series for the symplectic group of degree 2. We obtain, in this way, the meromorphic continuation of the Dirichlet series to $$\mathbb {C}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>C</mml:mi> </mml:math> as a corollary. In the case of the $$E_8$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>E</mml:mi> <mml:mn>8</mml:mn> </mml:msub> </mml:math> lattice, we are able to further deduce a precise functional equation for the Dirichlet series.