Understanding the Finite Size and Surface Relaxation Effects on the Surface States of Bi ₂ Se ₃ Family Topological Insulators

Topological insulators (TIs) of the Bi2Se3 family exhibit a topological phase transition from three-dimensional (3D) to two-dimensional (2D) TIs in thin films with decreasing thickness. Understanding the driving force of this transition is critical for the applications of TIs in nanodevices. Herein, we investigate the finite-size effects on bulk band inversion and the structural relaxation effects on the surface states within the Bi2Se3 family via first-principles calculations. Thin films exposing the three lowest-energy surfaces are modeled by 2D slabs with tunable thicknesses. We propose that the thickness dependence of the topological phase originates from electron confinement created by surface cuts. The increase in film thickness then counteracts these confinement effects, resulting in a monotonically decreasing band gap evaluated at the spin–orbit decoupled level. This dependence of the bulk gap on the thickness is found to be consistent for various surface slabs. We utilize this relationship to predict the required thickness for maintaining the 3D TI phase and 2D surface states. Our findings underscore the importance of electron delocalization in determining the topological phase of TI thin films. In addition, the actual manifestation of topological surface states on the side surfaces is affected significantly by the coexisting dangling bonds produced by surface cuts. Therefore, surface relaxation plays a crucial role in disentangling the trivial and nontrivial surface states.