Systematic evaluation of attention mechanisms in transformer models for De Novo UTS-driven silk protein sequence design

Abstract Protein sequence generation has emerged as a central theme in computational biology, with silk proteins receiving attention due to their superior mechanical properties compared to synthetic materials. However, systematic evaluation of transformer architectures for silk protein sequence generation remains limited. In this study, we conducted a comparative analysis of five attention mechanisms in transformer architectures optimized for silk protein crystalline region generation, validated by steered molecular dynamics simulations. Among the tested mechanisms, convolutional attention demonstrated the best performance, achieving a correlation coefficient of 0.91 between target and simulated ultimate tensile strength values. Attention map analysis further revealed that convolutional attention effectively captured local sequence dependencies critical for β-sheet formation. These findings provide a foundation for data-driven design of protein-based materials with tunable mechanical properties across a broad range.