Adaptive DQN-Based Multimodal Oral English Teaching Platform for Enhanced Student Performance Evaluation and Strategy Optimization

This paper aims to build an oral English teaching platform, accurately evaluate students’ oral ability through multimodal data analysis, and optimize teaching strategies and improve teaching effects by combining adaptive DQN (Deep Q-Network). This paper combines three-modal features: speech, text and image. Speech features are extracted through MFCC (Mel-Frequency Cepstral Coefficients), text features are extracted through BERT (Bidirectional Encoder Representations from Transformers), and image features are extracted through ResNet-50 (Residual Network-50). Multimodal feature vectors are fused through vector concatenation, and XGBoost is used to classify English-speaking ability. Combining the oral ability classification results with student behavior characteristics, the adaptive DQN algorithm is used to optimize the personalized learning strategy and dynamically adjust the learning content and difficulty. The classification accuracy of the Voice-Text-Image modality combination in this paper reached 96.3%. The platform scores students’ oral ability through the IELTS (International English Language Testing System) speaking assessment system. The experimental results show that at the 12th month of intervention, the IELTS speaking score of students on this platform was 8 points, while that of students on the traditional platform was 6 points. In multiple simulated dialogue scenarios, the average pronunciation accuracy and average speaking speed of students on this platform reached 91.8% and 129 words/min, respectively, higher than those of traditional platforms. The oral teaching platform combining adaptive DQN and multimodal analysis can improve students’ oral ability. This paper provides an innovative teaching model for oral English education, with high application value and promotion potential.