Competing ferromagnetic and antiferromagnetic phases on the frustrated Ising honeycomb lattice

We investigate the frustrated J 1 – J 2 – J 3 Ising model on the honeycomb lattice, featuring first- and second-neighbor ferromagnetic couplings ( J 1 > 0 and J 2 > 0 ) and third-neighbor antiferromagnetic interactions ( J 3 < 0 ). Using the cluster mean-field method, we analyze the phase transitions in the regime 1 / 2 < J 2 / J 1 ≤ 1 , where ferromagnetic and antiferromagnetic phases compete. Our results reveal that near the strongly frustrated limit J 3 / J 1 = − 1 , the system exhibits order-by-disorder state selection, tricritical and bicritical behavior, critical endpoints, and two successive phase transitions. The ferromagnetic–paramagnetic transition remains second order across the entire interaction range, whereas the antiferromagnetic–paramagnetic boundary shows a richer behavior, including both first- and second-order transitions as well as tricriticality. Increasing the second-neighbor coupling J 2 / J 1 narrows the range of J 3 / J 1 where first-order antiferromagnetic–paramagnetic transitions occur; beyond a certain threshold, only second-order order–disorder transitions persist. Consequently, the tricritical point shifts toward J 3 / J 1 ≈ − 1 as J 2 / J 1 increases, culminating in a bicritical point where the antiferromagnetic, ferromagnetic, and paramagnetic phases meet. • The J 1 - J 2 - J 3 Ising honeycomb lattice is studied using the cluster mean-field method. • The ferromagnetic-paramagnetic phase transitions are second-order. • The antiferromagnetic-paramagnetic phase boundary exhibits tricriticality. • For a large enough J 2 / J 1 only second-order order–disorder phase transitions occur.