SilentSpeak: A Proposed Multilingual Deep Learning Frame work for Lip Reading in English, Hindi, and Marathi to Assist Individuals with Speech and Hearing Impairments

Lip reading, or visual speech recognition, can recover spoken content when audio is missing, unusable or inaudible, which is valuable for people with hearing or speech impairments. Most published research targets Eng lish, and the authors' literature search found no publicly documented word-level lip-reading dataset or system for Hindi or Marathi. This paper proposes SilentSpeak, a conceptual framework for word-level lip reading in English, Hindi and Marathi that is designed to run on a standard webcam without an internet connection. The proposed pipeline locates the lips with MediaPipe Face Mesh, describes each frame with a small convolutional network, and models the mouth movement over time with a bidirectional long short-term memory network. An English base model trained on the GRID corpus is to be adapted to Hindi and Marathi through transfer learning, by freezing the convolutional layers and retraining the recurrent and output layers on small pilot vocabularies. The framework also proposes confidence scoring with a top-three fallback, an emergency-keyword alert and offline speech output. The paper reviews the rele vant literature, identifies the research gap, specifies the methodology and system architecture, and outlines a plan for evaluation. This is a proposed research direction, and no experimental results are claimed in this paper.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24286303
Primary Topic
Speech and Audio Processing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

SilentSpeak: A Proposed Multilingual Deep Learning Frame work for Lip Reading in English, Hindi, and Marathi to Assist Individuals with Speech and Hearing Impairments

Mahek Shaukat Patel, Priyanka Pramod Kurde, Mrs. Aarti Patil, Sakshi Ganesh Muley
2 links
Zenodo (CERN European Organization for Nuclear Research)
Speech and Audio Processing
article

SilentSpeak: A Proposed Multilingual Deep Learning Frame work for Lip Reading in English, Hindi, and Marathi to Assist Individuals with Speech and Hearing Impairments

Mahek Shaukat Patel, Priyanka Pramod Kurde, Mrs. Aarti Patil, Sakshi Ganesh Muley
article en

Abstract

Lip reading, or visual speech recognition, can recover spoken content when audio is missing, unusable or inaudible, which is valuable for people with hearing or speech impairments. Most published research targets Eng lish, and the authors' literature search found no publicly documented word-level lip-reading dataset or system for Hindi or Marathi. This paper proposes SilentSpeak, a conceptual framework for word-level lip reading in English, Hindi and Marathi that is designed to run on a standard webcam without an internet connection. The proposed pipeline locates the lips with MediaPipe Face Mesh, describes each frame with a small convolutional network, and models the mouth movement over time with a bidirectional long short-term memory network. An English base model trained on the GRID corpus is to be adapted to Hindi and Marathi through transfer learning, by freezing the convolutional layers and retraining the recurrent and output layers on small pilot vocabularies. The framework also proposes confidence scoring with a top-three fallback, an emergency-keyword alert and offline speech output. The paper reviews the rele vant literature, identifies the research gap, specifies the methodology and system architecture, and outlines a plan for evaluation. This is a proposed research direction, and no experimental results are claimed in this paper.

Zenodo (CERN European Organization for Nuclear Research)
Reduced inequalities
Speech and Audio Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon