Multimodal CNN-GRU Approach for Job Interview Assessment: Integrating Facial Expressions and Speech Analysis

Thorough evaluation of job interview abilities demands a complete review of verbal communication signals together with non-verbal expression signals. This work demonstrates a CNN-GRU model which analyzes candidate evaluations from facial characteristics together with speech indicators. The CNN identifies fundamental facial signals comprised of engagement levels and eye contact along with smile intensity yet the GRU examines voice attributes containing tonal qualities and speech speed and breathing pauses. The model operates on interview recording data through training before receiving assessment through common classification measurements. The experimental findings demonstrate that the CNN-GRU combination outperforms singular CNN and GRU networks by reaching 88.6% accuracy and $87.5 \%$ precision and $86.8 \%$ recall while providing $87.1 \%$ F1-score. The traditional machine learning SVM model reaches an accuracy rate of $74.2 \%$ which proves inferior when compared to the multimodal approach’s higher $88.6 \%$ accuracy. The confusion matrix examination shows that misclassification happens minimally even when dealing with intermediate cases. Classification effectiveness insights stem from facial and speech features through Grad-CAM and attention visualizations methods. Research results demonstrate how multimodal techniques produce an automated interview assessment framework which reaches high levels of objective accuracy and reliability. Future research needs to study bigger datasets with advanced interpretability methods to improve practical applications of this method.