Music teaching evaluation using multimodal deep reinforcement learning

This paper presents a Multi-modal Music Teaching Evaluation System (MMTES) that combines Deep Reinforcement Learning (DRL) with Retrieval-Augmented Generation (RAG) to provide personalized, accurate, and adaptive feedback in music education. Conventional evaluation approaches often rely on unimodal data, limiting their ability to capture the complexity of multimodal teaching environments. In contrast, MMTES integrates synchronized audio, video, symbolic music notation, and textual feedback, offering a holistic assessment of both teaching effectiveness and student performance. Within the system, a DRL agent adaptively refines evaluation strategies by maximizing cumulative rewards aligned with learning progress, while an RAG-enhanced language model retrieves pedagogical knowledge to generate context-aware evaluation and feedback. To ensure transparency and trust, a blockchain module is incorporated for secure data management. Experimental results on both benchmark and custom multimodal datasets demonstrate that MMTES surpasses state-of-the-art baselines in evaluation accuracy, contextual relevance, and adaptability. These results highlight the system’s potential as an intelligent, scalable, and reliable tool for advancing music education in real-world settings.