TransformerCas9: an interpretable multi-scale Transformer with hierarchical attention for CRISPR/Cas9 sgRNA activity prediction

Background Accurate prediction of single-guide RNA (sgRNA) activity is critical for effective CRISPR/Cas9 genome editing. Despite substantial advances in computational modeling, existing approaches remain limited by suboptimal predictive performance, restricted generalizability across diverse Cas9 variants, and insufficient interpretability. Results We present TransformerCas9, an interpretable Transformer-based framework that integrates multi-scale K-mer encoding with a hierarchical attention mechanism to simultaneously capture local sequence features and long-range contextual dependencies, offering a new framework for accurate sgRNA activity prediction. Systematic analyses of the contributions of key architectural components and biological features further demonstrate the effectiveness of the proposed framework. On the HT_Cas9_Test benchmark dataset, TransformerCas9 achieves a Spearman correlation coefficient of 0.788 and a Pearson correlation coefficient of 0.790, surpassing all evaluated baseline models. Furthermore, across seven datasets encompassing wild-type, high-fidelity, and PAM-flexible SpCas9 variants, TransformerCas9 consistently outperforms six state-of-the-art approaches, achieving the highest performance on five datasets and the best overall ranking among all evaluated methods. Interpretability analyses leveraging intrinsic attention weights, Integrated Gradients attribution paired with in silico perturbation, and complementary explanation approaches consistently highlight biologically relevant sequence determinants associated with sgRNA activity, supporting the biological plausibility of the learned representations. Conclusions O ur work introduces TransformerCas9 as an accurate, robust, and interpretable framework for sgRNA activity prediction, which adopts multi-scale K-mer encoding, multi-head self-attention mechanism and a tailored MLA module to learn comprehensive sequence representations, providing a practical tool for CRISPR guide RNA design.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-18
DOI
https://doi.org/10.21203/rs.3.rs-11574502/v1
Primary Topic
CRISPR and Genetic Engineering
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

TransformerCas9: an interpretable multi-scale Transformer with hierarchical attention for CRISPR/Cas9 sgRNA activity prediction

Baoru Han, Haisong Liu, Jie Cao, Qingling Song et al.
3 links
Research Square
CRISPR and Genetic Engineering
preprint

TransformerCas9: an interpretable multi-scale Transformer with hierarchical attention for CRISPR/Cas9 sgRNA activity prediction

Baoru Han, Haisong Liu, Jie Cao, Qingling Song, J Li, Liuzhi Chen, Ziyu Zhai, Feng Liu, Jiaju Qi
preprint en

Abstract

Background Accurate prediction of single-guide RNA (sgRNA) activity is critical for effective CRISPR/Cas9 genome editing. Despite substantial advances in computational modeling, existing approaches remain limited by suboptimal predictive performance, restricted generalizability across diverse Cas9 variants, and insufficient interpretability. Results We present TransformerCas9, an interpretable Transformer-based framework that integrates multi-scale K-mer encoding with a hierarchical attention mechanism to simultaneously capture local sequence features and long-range contextual dependencies, offering a new framework for accurate sgRNA activity prediction. Systematic analyses of the contributions of key architectural components and biological features further demonstrate the effectiveness of the proposed framework. On the HT_Cas9_Test benchmark dataset, TransformerCas9 achieves a Spearman correlation coefficient of 0.788 and a Pearson correlation coefficient of 0.790, surpassing all evaluated baseline models. Furthermore, across seven datasets encompassing wild-type, high-fidelity, and PAM-flexible SpCas9 variants, TransformerCas9 consistently outperforms six state-of-the-art approaches, achieving the highest performance on five datasets and the best overall ranking among all evaluated methods. Interpretability analyses leveraging intrinsic attention weights, Integrated Gradients attribution paired with in silico perturbation, and complementary explanation approaches consistently highlight biologically relevant sequence determinants associated with sgRNA activity, supporting the biological plausibility of the learned representations. Conclusions O ur work introduces TransformerCas9 as an accurate, robust, and interpretable framework for sgRNA activity prediction, which adopts multi-scale K-mer encoding, multi-head self-attention mechanism and a tailored MLA module to learn comprehensive sequence representations, providing a practical tool for CRISPR guide RNA design.

Research Square
Chongqing Medical University (CN)
CRISPR and Genetic Engineering
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon