The explain-on-blackboard paradigm for interleaved visual-text reasoning
Abstract Interleaved visual-text chain-of-thought reasoning, generating intermediate visual artifacts to support deduction, is a promising paradigm for complex multimodal tasks. However, models suffer a representation tax causing systematic performance degradation when translating abstract reasoning into structured spatial visualizations. We attribute this to alignment bias in conventional supervised fine-tuning, which optimizes labels while neglecting synchronization between visual semantics and textual reasoning trajectories. To address this, we propose a pedagogical distillation framework grounded in the Explain-on-Blackboard (EoB) paradigm. Guided by multimedia learning and social constructivism, our method requires teacher models to generate temporally and semantically aligned teaching trajectories comprising a discourse channel for logical explanation and a canvas channel for incremental HTML/SVG construction. A dynamic-rubric quality assurance mechanism with multi-stage filtering retains only high-fidelity trajectories for distillation. To validate our approach, we construct Cas-Bench, a 1,000-problem multidisciplinary protocol. Experiments show an 8B-parameter model fine-tuned on 4,000 distilled trajectories achieves a 69.76% relative improvement on Cas-Bench over baselines across MMMU, MMMU-Pro, MathVision, and MathVista benchmarks.
Authors
- Bihui Yu (ORCID: https://orcid.org/0000-0001-8614-7350)
- Taishan Zeng (ORCID: https://orcid.org/0000-0001-9153-2878)
- Zhaojin Fu (ORCID: https://orcid.org/0009-0003-4917-6867)
- Linzhuang Sun (ORCID: https://orcid.org/0000-0003-0363-3607)
- Jianwei Cao (ORCID: https://orcid.org/0000-0002-6664-7080)
- Sizhe Qiu
Institutions
- South China Normal University (CN)
- Chinese Academy of Sciences (CN)
- Shenyang Institute of Computing Technology (China) (CN)
- University of Chinese Academy of Sciences (CN)
- University of Science and Technology Beijing (CN)
Publication Details
- Journal
- Multimedia Systems
- Published
- 2026-11-01
- DOI
- https://doi.org/10.1007/s00530-026-02735-7
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00