Unconventional hybrid-order topological insulators

Exploring novel topological matters with exotic quantum states has always been a core issue in the field of condensed matter physics, which can update the understanding of topological phases and broaden the classification of topological materials. Here, we report a class of unconventional hybrid-order topological insulators (HyOTIs), which simultaneously host various higher-order topological states in a single band gap. Such topological states exhibit a unique bulk-boundary correspondence that is different from the well-known first-order topological states, higher-order topological states, and the coexistence of both. We develop a generic surface theory to precisely capture them and discover a three-dimensional unconventional HyOTI protected by inversion symmetry, which renders both helical and corner topological states and exhibits an excellent bulk-edge-corner correspondence. By adjusting the parameters of the system, we also observe the nontrivial phase transitions between the inversion-symmetric HyOTI and other conventional phases. We further propose a circuit-based experimental scheme to detect these interesting results. Particularly, we demonstrate that a modified tight-binding model of bismuth can support the unconventional HyOTI, suggesting a possible route for its material realization. This work significantly advances the research of hybrid topological states in both theory and experiment.