A Multimodal Fusion and Deep Learning‐Based Behaviour Recognition Model for Assessing Executive Function in Young Children

ABSTRACT Measurement of executive function (EF) in early childhood is critical for the identification of developmental pathways and the planning of timely interventions, but standard behavioural measures are frequently subjective, time‐consuming and of limited scalability. Standard unimodal methods that depend exclusively on video or audio are not robust enough to be able to capture the richness of children's cognitive and behavioural signals. To overcome these limitations, we introduce a multimodal fusion and deep learning‐based behaviour recognition approach that combines video, audio and sensor‐derived features for the automated EF assessment. The novelty of this research is to combine heterogeneous data streams using multimodal fusion and provide a richer and more accurate representation of cognitive states in contrast to single‐modality baselines. Experimental findings show that the model proposed by us achieves 97.75% accuracy, with precision (0.9809), recall (0.976) and F1‐score (0.9781) but with low RMSE and high correlation with human‐rated EF scores. Comparative analysis identifies that our framework is much superior to baseline models like ResNet50 + LSTM (53.37%), Bi‐LSTM with MFCCs (25.72%) and MS‐RRBR (94.94%), validating the advantage of multimodal integration. The method yields consistent and strong classification across levels of cognition, backed up by ROC/PR curves with nearly perfect AUC values and confusion matrix evaluation with negligible misclassification. Aside from improved performance, the methodology presents a scalable and unbiased instrument for early cognitive testing, with possible uses in educational technology and clinical applications. Overall, this research provides an effective and explainable AI‐based approach for progressing personalised learning and early child development assessment.