Signatures of magnetic and topological hall effects in Mn2BiSbTe5: an anomalous low-temperature transition

• Mn 2 BiSbTe 5 single crystals were successfully synthesized using the self-flux method. • X-ray diffraction (XRD) and structural analysis confirm a highly ordered layered structure with a nonuple-layer stacking. • Magnetization measurements reveal an antiferromagnetic transition at ∼19 K, with a clear zero-field-cooled (ZFC) and field-cooled (FC) bifurcation, indicating cluster spin-glass behavior confirmed through AC susceptibility. • Transport studies show an anomalous Hall effect (AHE) and topological Hall effect (THE), confirming Berry curvature-driven transport and nontrivial topology. • Mn 2 BiSbTe 5 provides a new platform for realizing topological quantum transport phenomena, making it a strong candidate for spintronic and quantum material applications. We report a comprehensive investigation of Mn 2 BiSbTe 5 , a new member of the Mn-based tetradymite family of magnetic topological insulators (MTIs). High-quality single crystals were grown and characterized using structural, spectroscopic, and transport probes. Raman spectroscopy reveals vibrational modes consistent with the crystal symmetry, while X-ray photoelectron spectroscopy (XPS) confirms the chemical composition and elemental valence states. Magnetization and magneto-transport measurements identify a magnetic transition below 20 K, with a sharp anomaly in the derivative of magnetization marking the onset of long-range order. A bifurcation between zero-field-cooled and field-cooled magnetization suggests cluster spin-glass behavior. Hall resistivity measurements further uncover a topological Hall effect, attributed to real-space spin chirality and nontrivial topological transport. The tunable interplay between magnetic ordering and topological response establishes Mn 2 BiSbTe 5 as a promising platform for realizing axion insulator phases, anomalous quantum Hall states, and topology-driven spin transport, advancing the development of bulk MTIs for quantum electronics and spintronics.