$Σ$-Attention: A Transformer-based operator learning framework for self-energy in strongly correlated systems

We introduce $Σ$-Attention, a Transformer-based operator-learning framework to address a key computational challenge in correlated materials. Our approach utilizes an Encoder-Only Transformer as an ansatz to approximate the self-energy operator of strongly correlated electronic systems. By creating a batched dataset that combines results from three complementary approaches: many-body perturbation theory, strong-coupling expansion, and exact diagonalization, each effective in specific parameter regimes, $Σ$-Attention is applied to learn a universal approximation for the self-energy operator that is valid across all regimes. This hybrid strategy leverages the strengths of existing methods while relying on the transformer's ability to generalize beyond individual limitations. More importantly, the scalability of the Transformer architecture allows the learned self-energy to be extended to systems with larger sizes, leading to much improved computational scaling. Using the 1D Hubbard model, we demonstrate that $Σ$-Attention can accurately predict the Matsubara Green's function and hence effectively captures the Mott transition at finite temperature. Our framework offers a promising and scalable pathway for studying strongly correlated systems with many possible generalizations.