Interpretable and lightweight multimodal Transformer assessment of English education ecosystem health
How can the health of an English classroom ecosystem be assessed from routine recordings in a way that is accurate, traceable to observable behaviour and inexpensive enough to run inside a school? The question is harder than it looks, because what is taught, how it is delivered and how learners respond are dynamically coupled: the difficulty of the material shapes delivery, and delivery in turn shapes learner behaviour. Single-modality models and shallow feature concatenation capture only fragments of that interplay. To answer the question, we build a multimodal Transformer that encodes textual, acoustic and visual streams through modality-specific front ends, learns their mutual dependence through bidirectional cross-modal attention, and merges them through a learned gate to predict a five-dimensional Ecosystem Health Index whose scoring rubric, rater protocol and entropy-based weighting are fully specified. A gradient-weighted attribution scheme turns attention into modality-decomposed, temporally localised explanations, which show textual features dominating healthy sessions and video-based disengagement cues taking over in degraded ones. A joint distillation and structured pruning pipeline compresses the network from 23.8 M to 2.9 M parameters, an 8.2-fold reduction that retains 88.2% of the teacher model’s RMSE-based performance. On 6,142 sessions recorded in 20 institutions the full model reaches an RMSE of 0.0431 ± 0.0009 and a coefficient of determination of 0.923 ± 0.004 across five random seeds, ahead of unimodal, early-fusion and prior cross-modal baselines under paired significance testing. Error rises by 13.5% to 34.1% once instructors, institutions, provinces, course types or educational levels are held out, and we treat that gap as the honest boundary of the present evidence rather than as noise. Device-level benchmarks on Jetson Nano and Raspberry Pi hardware, together with an explicit responsible-use protocol that rules out punitive teacher evaluation, complete the account.
Authors
- Meiqin Lv
- You Wang
- Dong Han
Institutions
- Northeast Normal University (CN)
- Jilin University (CN)
- Wuhan Vocational College of Software and Engineering (CN)
- Wuhan Open University (CN)
Publication Details
- Journal
- Scientific Reports
- Published
- 2026-11-07
- DOI
- https://doi.org/10.1038/s41598-026-77249-3
- Primary Topic
- Educational Technology and Pedagogy
- Type
- article
- Field-Weighted Citation Impact
- 0.00