Leveraging Confidence Analysis and Classification Using BiLSTM for Verbal Evaluations

Understanding the emotion and confidence level of a respondent’s answer helps formulate the response as well as the next question in an oral examination or viva. Automatic detection of these parameters is essential to develop appropriate next statements, provided the relevant tone of a Viva chatbot curated for assistance and analysis of human speech in interview preparations. Supporting this motivation, this work analyzes the effectiveness of using various spectral attributes such as Mel Frequency Cepstral Coefficients (MFCC), Chroma, Mel Spectrogram & Spectral Contrast alongside CNN, RNN, and Bi-LSTM networks to identify confidence & emotion in speech signals. It is observed that MFCC features along with a BiLSTM network provide the best results with 63.76% test accuracy and 58.82% train accuracy by considering minimal variance.