Audio-based Age, Gender & Emotion detection

Objectives: The purpose of this study is to produce an automated system using deep learning that can guess the gender, age, and mood of a speaker from their speech. The purpose is to connect people to computers better, support early diagnostics in medicine, and make automated customer service more caring by responding in a flexible way using a person’s speech. Methods: Three specialised deep learning models will be applied to the study: an LSTM-based classifier predicting gender, a CNN-based model classifying age based on the Mozilla Common Voice audio files, and a fine-tuned Deepsight model to predict the emotion, based on a mixture of CREMA-D, RAVDESS, SAVEE, and TESS data sets. Such data provides speech at various genders and ages with different emotions that include happiness, sadness, anger, and neutrality. Normalisation of the audio levels, noise removal, and the extraction of features on the audio data with the help of Mel-Frequency Cepstral Coefficients (MFCCs) were performed as preprocessing steps. Cross-validation was used to evaluate the models and test on unseen speech data as a way of measuring generalizability. Findings: The proposed system attained a gender predictive accuracy of 96.3 and an F1-score of 0.95. To classify speaking age into 10-year age groupings, the model ranked with an accuracy of 87.4%. The main emotional states were identified by the emotion recognition module with an accuracy of 91.2 per cent. K-fold cross-validation confirmed the performance, and the performance was also re-confirmed on practical audio samples. The combined pipeline model showed improved consistency and reduced computation complexity in comparison to the methods used when every model was run separately. Novelty: Instead of choosing one factor at a time, the present work looks at gender, age, and emotions all at once from a single voice sample. Since the end-to-end model pipeline is simple and time-saving, it helps build real-time virtual assistants, telemedicine tools, and programs focused on feelings. Keywords: Speech Processing, Deep Learning, LSTM, Voice Recognition, Multimodal Voice Profiling