Vortex phase transition in spin-orbit coupled Lee-Huang-Yang condensates

The interplay of quantum fluctuation through the Lee-Huang-Yang (LHY) correction and spin-orbit coupling can modify the stability of the ultradilute Bose-Bose mixtures and result in rich nonlinear physics. Here, under the action of spin-orbit coupling, LHY correction, and rotation, the ground state and vortex excitation of spin-orbit coupled Bose-Einstein condensates (BECs) with LHY correction trapped in a rotating potential are studied. Stable isotropic and anisotropic ground states with zero momentum (in the zero-momentum phase) and nonzero momentum (in the plane wave phase) are obtained. It is shown that anisotropic ground states are persistent near the phase boundary. Particularly, unpolarized and polarized vortex states are predicted. The phase transition condition, the effective mass and quadrupole surface mode describing the anisotropic (deformation) of the condensates, and the critical rotation frequency for vortex excitation are provided analytically. Owing to the action of spin-orbit coupling, the effective mass of the condensates is spatially dependent and asymmetric in two phases, resulting in distinct quadrupole modes, deformations, and vortex excitations in the two phases, especially when the LHY correction is present. The LHY correction promotes the system in the zero-momentum phase with unpolarized vortex states, while the spin-orbit coupling promotes the emergence of a plane wave phase with polarized vortex states. The phase transition and vortex excitation can be well manipulated by spin-orbit coupling, Raman coupling, LHY correction, and external potential.