Cation-Size-Controlled Structural and Magnetic Ground State Transitions in Co2+-Based Honeycomb-Derived Oxides Na0.5 RE 0.5Co2TeO6 ( RE = La, Nd, Sm, Eu, Tb, Dy, Yb, and Y)

Abstract Honeycomb lattices containing spin–orbit-entangled Co2+ ions with effective Jeff = 1/2 moments provide promising platforms for enhancing strong quantum fluctuations and thus realizing potential quantum spin-liquid states. Here, we report a series of honeycomb-derived tellurates, “Na0.5RE0.5Co2TeO6” (NRECTO, RE = La, Nd, Sm, Eu, Tb, Dy, Yb, and Y). The increased size mismatch between Na+ and RE3+ drives an orthorhombic-to-hexagonal structural transition for NRECTO, accompanied by reconstruction in the connectivity of the [Co2TeO6]2– double-rutile chains and transformation in hexagonal channel topologies. High-resolution synchrotron X-ray diffraction and neutron powder diffraction reveal that both structures retain similar three-dimensional Co2+ topologies derived from AAA-stacked buckled honeycomb layers. Magnetic measurements revealed that orthorhombic NLaCTO exhibits a long-range antiferromagnetic order below 13 K, as confirmed by DC and AC susceptibility and a λ-type heat-capacity anomaly, whereas hexagonal NYCTO exhibits glassy magnetic freezing below 11 K, characterized by frequency-dependent AC susceptibility and slow spin dynamics. Despite comparable Co2+–O–Co2+ exchange pathways and Co-topologies are retained in both structures, their contrasting magnetic behavior demonstrates that subtle structural differences can strongly reshape the magnetic energy landscape. These results establish chemical-unit substitution as an effective strategy for controlling magnetism in Co2+-based honeycomb oxides.