Vowel Recognition in Lipreading Using Geometric Features

Lipreading, the ability to interpret spoken words through lip movements, faces significant challenges in real-world applications, such as distinguishing vowels with similar lip shapes under natural lighting and variable orientations. This study aims to address the problem of accurately recognizing vowels (a, i, u, e, o) by focusing on geometric lip angle features. The research leverages a 1D Convolutional Neural Network (CNN) model, chosen for its ability to effectively capture sequential data patterns, with three variations: a single Conv1D layer, two Conv1D layers, and three Conv1D layers. The dataset comprises 140 videos of two speakers recorded under realistic conditions, with angular features extracted from outer lip landmarks (points 49–60). Results show that while the training accuracy approached 100%, the models struggled with real-world generalization, correctly recognizing the vowel "a" but facing difficulty in distinguishing others, such as "i" and "e," due to their similar lip shapes. This highlights the complexity of using angular features alone in vowel recognition. By addressing these challenges, this study contributes to the development of robust lipreading systems with potential applications in noisy environments, security, and assistive technologies for the hearing-impaired.