The explain-on-blackboard paradigm for interleaved visual-text reasoning

Abstract Interleaved visual-text chain-of-thought reasoning, generating intermediate visual artifacts to support deduction, is a promising paradigm for complex multimodal tasks. However, models suffer a representation tax causing systematic performance degradation when translating abstract reasoning into structured spatial visualizations. We attribute this to alignment bias in conventional supervised fine-tuning, which optimizes labels while neglecting synchronization between visual semantics and textual reasoning trajectories. To address this, we propose a pedagogical distillation framework grounded in the Explain-on-Blackboard (EoB) paradigm. Guided by multimedia learning and social constructivism, our method requires teacher models to generate temporally and semantically aligned teaching trajectories comprising a discourse channel for logical explanation and a canvas channel for incremental HTML/SVG construction. A dynamic-rubric quality assurance mechanism with multi-stage filtering retains only high-fidelity trajectories for distillation. To validate our approach, we construct Cas-Bench, a 1,000-problem multidisciplinary protocol. Experiments show an 8B-parameter model fine-tuned on 4,000 distilled trajectories achieves a 69.76% relative improvement on Cas-Bench over baselines across MMMU, MMMU-Pro, MathVision, and MathVista benchmarks.

Authors

Institutions

Publication Details

Journal
Multimedia Systems
Published
2026-11-01
DOI
https://doi.org/10.1007/s00530-026-02735-7
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

The explain-on-blackboard paradigm for interleaved visual-text reasoning

Bihui Yu, Taishan Zeng, Zhaojin Fu, Linzhuang Sun et al.
Multimedia Systems
Multimodal Machine Learning Applications
article

The explain-on-blackboard paradigm for interleaved visual-text reasoning

Bihui Yu, Taishan Zeng, Zhaojin Fu, Linzhuang Sun, Jianwei Cao, Sizhe Qiu
article en

Abstract

Abstract Interleaved visual-text chain-of-thought reasoning, generating intermediate visual artifacts to support deduction, is a promising paradigm for complex multimodal tasks. However, models suffer a representation tax causing systematic performance degradation when translating abstract reasoning into structured spatial visualizations. We attribute this to alignment bias in conventional supervised fine-tuning, which optimizes labels while neglecting synchronization between visual semantics and textual reasoning trajectories. To address this, we propose a pedagogical distillation framework grounded in the Explain-on-Blackboard (EoB) paradigm. Guided by multimedia learning and social constructivism, our method requires teacher models to generate temporally and semantically aligned teaching trajectories comprising a discourse channel for logical explanation and a canvas channel for incremental HTML/SVG construction. A dynamic-rubric quality assurance mechanism with multi-stage filtering retains only high-fidelity trajectories for distillation. To validate our approach, we construct Cas-Bench, a 1,000-problem multidisciplinary protocol. Experiments show an 8B-parameter model fine-tuned on 4,000 distilled trajectories achieves a 69.76% relative improvement on Cas-Bench over baselines across MMMU, MMMU-Pro, MathVision, and MathVista benchmarks.

Multimedia SystemsVol. 32(11)
South China Normal University (CN), Chinese Academy of Sciences (CN), Shenyang Institute of Computing Technology (China) (CN), University of Chinese Academy of Sciences (CN), University of Science and Technology Beijing (CN)
Quality education
Openalex Percentile: Top 68%
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon