Spin dynamics of an easy-plane Dirac spin liquid in a frustrated XY model: Application to honeycomb cobaltates

Recent work has shown that the honeycomb lattice spin-$\frac{1}{2}\phantom{\rule{4pt}{0ex}}{J}_{1}\text{\ensuremath{-}}{J}_{3}$ XY model, with nearest-neighbor ferromagnetic exchange ${J}_{1}$ and frustration induced by third-neighbor antiferromagnetic exchange ${J}_{3}$, may be relevant to a wide range of cobaltate materials. We explore a variational Monte Carlo (VMC) study of Gutzwiller projected wave functions for this model and show that an easy-plane Dirac spin liquid (DSL) is a viable ``parent'' state for the competing magnetic orders observed in these materials, including ferromagnetic, zigzag, spiral, and double zigzag orders at intermediate frustration, and show that such broken-symmetry states can be easily polarized by a weak in-plane magnetic field consistent with experiments. We formulate a modified parton theory for such frustrated spin models, and explore the potential instabilities of the DSL due to residual parton interactions within a random phase approximation (RPA), both at zero magnetic field and in a nonzero in-plane field. The broken-symmetry states which emerge in the vicinity of this Dirac spin liquid include ferromagnetic, zigzag, and incommensurate spiral orders, with a phase diagram which is consistent with VMC and density matrix renormalization group studies. We calculate the dynamical spin response of the easy-plane DSL, including RPA corrections, near the boundary of the ordered states, and present results for THz spectroscopy and inelastic neutron scattering, at zero field as well as in an in-plane magnetic field, and discuss experimental implications.