Deep learning and attention mechanisms in RNA secondary structure prediction: A critical survey

The secondary structure of ribonucleic acid (RNA) plays a key role in understanding gene regulation, cellular processes, and the development of new treatments. Traditional thermodynamic methods, especially those using Minimum Free Energy (MFE) algorithms, have provided a reliable physics-based approach for predicting RNA structures. Although these methods remain important, there is increasing interest in using deep learning models to detect new structural patterns, such as pseudoknots and long-range interactions, in large RNA datasets. Building on thermodynamic principles, these models aim to extend current knowledge and offer new ways to study RNA structure and function. In particular, attention-based transformer models are effective at capturing both short- and long-distance relationships, making them well-suited for modeling complex RNA sequences. This review highlights recent advances in RNA secondary structure prediction using transformer-based approaches, focusing on key models such as E2EFold, ATTFold, RNAformer, and DEBFold. It also discusses current challenges, future research directions, and the impact of attention-based deep learning on the field of RNA structural bioinformatics.