Designing Temporally Grounded Multimodal Learning Interfaces for Verifiable Interaction

Abstract Textbooks, diagrams, and recorded lectures often provide complementary accounts of the same concept. Conversational learning assistants can retrieve relevant material, yet learners still need a way to inspect the original source of a generated statement. This preprint presents the system architecture and interaction design of a multimodal retrieval-augmented learning prototype that indexes textbook passages, descriptions of textbook figures, and timestamped descriptions of sampled lecture-video frames. Its central contribution is a provenance-to-viewer mapping that connects retrieved evidence with inspectable document pages and video timestamps through an explicit source registry. The accompanying Python and Streamlit research prototype incorporates dense semantic retrieval, an evidence-first split-pane interface, structured evidence controls, and handling for missing or invalid source references. The paper distinguishes implemented mechanisms from design intentions and identifies limitations concerning speech transcription, claim-level attribution, and end-to-end navigation verification. We describe this approach as verifiable interaction: enabling learners to move from AI-generated explanations or retrieved references toward original materials for independent inspection. This is a system and interaction-design contribution, not a controlled evaluation of retrieval accuracy, usability, or learning outcomes. Research context A preliminary version of this work was included in Spotlight Presentations II at the AIED 2026 Workshop on Designing and Evaluating Next-Generation Learning Interfaces. This October 2026 revision is an independent system and interaction-design preprint, not a separately peer-reviewed proceedings publication. Code repository: https://github.com/William-Turing/multimodal-rag-verifiable-tutor Workshop programme: https://extendedhorizon.com/aied2026ws-ai-hci-ls/program/