Existence and nonexistence of least energy positive solutions to critical Schrödinger systems with Hardy potential

Abstract We are concerned with the following coupled Schrödinger system with a Hardy potential in the critical case \begin{align*} \begin{cases} -\Delta u_{i}-\frac{\lambda_{i}}{|x|^2}u_{i}=|u_i|^{2^*-2}u_i+\sum_{j\neq i}^{3}\beta_{ij}|u_{j}|^{\frac{2^*}{2}}|u_i|^{\frac{2^*}{2}-2}u_i, ~x\in \mathbb{R}^N,\\ u_i\in D^{1,2}(\mathbb{R}^N),\,\, N\geq 3,\,\, i=1,2,3, \end{cases} \end{align*} where $2^*=\frac{2N}{N-2}$ , $\lambda_i\in (0,\Lambda_N), \Lambda_N:= \frac{(N-2)^2}{4}$ , and $\beta_{ij}=\beta_{ji}$ for i ≠ j . By virtue of variational methods, we establish the existence and nonexistence of least energy solutions for the purely cooperative case ( $\beta_{ij} \gt 0$ for any i ≠ j ) and the simultaneous cooperation and competition case ( $\beta_{i_{1}j_{1}} \gt 0$ and $\beta_{i_{2}j_{2}} \lt 0$ for some $(i_{1}, j_{1})$ and $(i_{2}, j_{2})$ ). Moreover, it is shown that fully nontrivial ground state solutions exist when $\beta_{ij}\ge0$ and $N\ge5$ , but NOT in the weakly pure cooperative case ( $\beta_{ij} \gt 0$ and small, i ≠ j ) when $N=3,4$ . We emphasize that this reveals that the existence of ground state solutions differs dramatically between $N=3, 4$ and higher dimensions $N\geq 5$ . In particular, the cases of N = 3 and $N\geq 5$ are more complicated than the case of N = 4 and the proofs heavily depend on the dimension. Some novel tricks are introduced for N = 3 and $N\ge5$ .