LESS-PseU: A Language Model-Enhanced Semistacking Framework for RNA Pseudouridine Site Prediction

Abstract Accurate prediction of RNA pseudouridine (Ψ) sites remains challenging because existing models often struggle to integrate interpretable handcrafted features with high-dimensional contextual representations from pretrained genomic language models. To address this issue, we propose LESS-PseU, a language model-enhanced semistacking framework that fuses the predicted probability features of primary learners with DNABERT2 embeddings for multiple species Ψ site prediction. Specifically, we first built three primary learners based on three kinds of hand-crafted features, including one-hot, k-mer, and nucleotide chemical property features. Instead of training a primary learner based on the embeddings extracted from DNABERT2, the embeddings were directly concatenated with the predicted probabilities of the three primary learners as the input features for a Logistic Regression meta-learner for final prediction. Under a leakage-aware cross-validation setting, our model was validated on the training sets of three species. Furthermore, on independent test sets from human, mouse, and yeast, LESS-PseU achieved AUC values of 0.9069, 0.8833, and 0.8820, respectively, which were higher than those of the compared models. A Friedman test indicated an overall difference (p = 0.029). Paired DeLong tests further showed that the AUC improvements over Meta-PseU were significant for the human and mouse data sets, while those over PseU-ST were significant for all three species. Ablation analysis, fusion-strategy comparison, and pretrained language-model comparison further demonstrate the effectiveness of the proposed framework. Additional evaluations on m5C and m7G benchmarks provided preliminary evidence that the framework may be applicable to other RNA modification prediction tasks.