Lip-Reading for Philippine Regional Language Classification of Cebuano and Ilocano

Automated lip-reading systems face significant challenges in multilingual environments, particularly in the Philippines, where dialects such as Cebuano and Ilocano have distinct phonetic-visual attributes. Existing methods mostly rely on audio and visual fusion for processing, while others, such as LipNet and LCSNet, focus on visual-only techniques using 3D Convolutional Neural Networks (3D CNN), Bidirectional LongShort Term Memory (BiLSTM), and Connectionist Temporal Classification (CTC). Although these systems have achieved excellent performance, such as LipNet with a 95.2 % accuracy, they lack the capability to classify multiple languages. This study introduces a lip-reading model inspired by LipNet and LCSNet that not only performs word-for-word prediction but also distinguishes between the Cebuano and Ilocano dialects. Data was gathered from 2000 videos of samples articulating lexicons in these languages, which were then evaluated using one general confusion matrix for the regional language classification gathering an accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{8 8 \%}$</tex>. Another two <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$10 \times 10$</tex> confusion matrices were also used to on measuring the accuracy of lexiconlevel classification resulting to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{6 4 \%}$</tex> for Cebuano and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{5 8 \%}$</tex> for Ilocano.