Advancing non-coding RNA annotation with RNA sequence foundation models: structure and function perspectives

Noncoding RNAs (ncRNAs) form the major part of the expressed transcriptome. These are critical in regulating gene expression and contributing to disease mechanisms, primarily through their complex secondary and tertiary structures. Despite advances in experimental and computational methodologies, accurate prediction of ncRNA structures and functions remains limited. Knowledge of RNA structures can not only answer basic biological questions but can also be of great help in the design of new types of drugs and therapies, emphasising the urgency for scalable and precise prediction tools. The experimental determination of the structure and function of ncRNAs remains costly and labour-intensive, which has prompted the development of computational approaches, including machine learning. Recent efforts are shifting towards using language modelling and developing generic foundation models (FM) to analyse RNA structure and function. These models leverage widely available unannotated sequence data to improve upon the generalisability and accuracy of earlier thermodynamic and deep learning techniques. A number of RNA FMs have been developed for diverse tasks such as secondary structure prediction, function annotation, RNA design. This review analyses current efforts in developing RNA FMs, evaluating their architectures, training strategies, and predictive capabilities. We highlight key trends, limitations, and outline open challenges, aiming to guide future research in RNA structure-function modelling.