Two-factor authentication based on keyword spotting and speaker verification

The subject matter of the article is the development and evaluation of a two-factor speaker authentication method based on voiceprintidentification and keyword spotting (KWS), designed for secure voice-based access in human-machine interfaces, especially forusers with limited mobility. The goal of the work is to create a method for managing speaker authentication using convolutionalneural networks (CNNs), comparing the efficiency of two widely used spectral feature extraction techniques – Mel-FrequencyCepstral Coefficients (MFCC) and Short-Time Fourier Transform (STFT) spectrograms. The following tasks were solved in thearticle: a model of a two-factor authentication method is proposed, which includes speaker identification and voicepassword recognition; the quality of MFCC and STFT spectrograms features is compared; the influence of the numberof epochs, CNN architecture and training parameters on the system accuracy is evaluated; the effect of the sampling rateon the performance of the models was investigated. The following methods are used: deep learning methods with CNNarchitecture, fine-tuning, MFCC, and STFT feature extraction, mathematical and statistical analysis of training efficiency,and system performance metrics. The following results were obtained: the method achieved 97.95% accuracy in speakeridentification using MFCCs after 60 training epochs, and 99.82% accuracy in voice password verification using the sameCNN structure after 20 epochs. The average accuracy of the entire authentication process was 98.75%. Moreover, using MFCCfeatures reduced training time by a factor of 23 and memory consumption by a factor of 7 compared to STFT spectrograms.Conclusions: the effectiveness of a two-factor voice authentication method that combines speaker identification by acousticvoice characteristics and voice password verification was implemented and studied. Further research directions include studyingthe impact of alternative spectral features (in particular, CQCC, GFCC, prosodic parameters) on improving accuracy and resistanceto spoofing. Special attention will be paid to optimizing the model for energy-efficient use on portable devices.