Nematic Order and Orbital Selective Mott State in a Partially Filled Kagome Flat Band

Coulomb interactions among charge carriers that occupy an electronic flat band (FB) can give rise to captivating phenomena such as quantum criticality, Mott-Hubbard states, and unconventional superconductivity at different FB filling fractions. Consequently, the search for new FB materials with tunable charge carrier filling and strong interactions is a central research theme to broaden our fundamental understanding of strongly interacting quantum systems and to realize materials with novel functionalities. In this work, we present experimental evidence obtained from temperature-dependent scanning tunneling microscopy measurements for various strongly correlated states that appear in partially occupied kagome FBs of <a:math xmlns:a="http://www.w3.org/1998/Math/MathML" display="inline"> <a:mrow> <a:msub> <a:mrow> <a:mi>Co</a:mi> </a:mrow> <a:mrow> <a:mn>1</a:mn> <a:mo>−</a:mo> <a:mi>x</a:mi> </a:mrow> </a:msub> <a:msub> <a:mrow> <a:mi>Fe</a:mi> </a:mrow> <a:mrow> <a:mi>x</a:mi> </a:mrow> </a:msub> <a:mi>Sn</a:mi> </a:mrow> </a:math> whose filling can be controlled by the Fe-doping level <c:math xmlns:c="http://www.w3.org/1998/Math/MathML" display="inline"> <c:mi>x</c:mi> </c:math> . At elevated temperatures ( <e:math xmlns:e="http://www.w3.org/1998/Math/MathML" display="inline"> <e:mi>T</e:mi> <e:mo>≥</e:mo> <e:mn>16</e:mn> <e:mtext> </e:mtext> <e:mtext> </e:mtext> <e:mi mathvariant="normal">K</e:mi> </e:math> ), we find evidence for rotation symmetry breaking across a broad doping range <h:math xmlns:h="http://www.w3.org/1998/Math/MathML" display="inline"> <h:mn>0.05</h:mn> <h:mo>&lt;</h:mo> <h:mi>x</h:mi> <h:mo>&lt;</h:mo> <h:mn>0.25</h:mn> </h:math> . Comparison with model calculations reveals that strong Coulomb interactions ( <j:math xmlns:j="http://www.w3.org/1998/Math/MathML" display="inline"> <j:mi>U</j:mi> <j:mo>&gt;</j:mo> <j:mn>100</j:mn> <j:mtext> </j:mtext> <j:mtext> </j:mtext> <j:mi>meV</j:mi> </j:math> ) blend the states of two <l:math xmlns:l="http://www.w3.org/1998/Math/MathML" display="inline"> <l:mn>3</l:mn> <l:mi>d</l:mi> </l:math> -orbital-derived FBs and impart a nematic order parameter. Introducing phase-sensitive lock-in measurements as a local scanning tunneling microscopy probe of incompressible states, we also find direct evidence for Hubbard bands in the partially filled kagome flat bands of samples with ideal Fe doping ( <n:math xmlns:n="http://www.w3.org/1998/Math/MathML" display="inline"> <n:mi>x</n:mi> <n:mo>=</n:mo> <n:mn>0.17</n:mn> </n:math> ). Results from Hubbard model calculations suggest the presence of an orbital-selective Mott state in a half-filled kagome flat band. Doping-dependent measurements show that the orbital-selective Mott state descends into a correlated metal phase upon electron and hole doping, marked by a sudden collapse of the spectral gap. Our observations demonstrate that the electronic ground state of the kagome FBs sensitively depends on the complex interplay between geometric frustration, strong Coulomb repulsion, orbital degeneracy, and filling fraction at different temperatures. More broadly, our research establishes kagome lattice materials as a unique platform for novel quantum states that arise from strong electronic correlations in a topologically nontrivial FB.