Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Networks and MediaPipe Framework

Few hearing Indonesians understand Bahasa Isyarat Indonesia (BISINDO), and most existing recognition systems classify raw images, which makes them sensitive to background, lighting and signer appearance. This study developed a real-time BISINDO alphabet recognition system that extracted 21 three-dimensional landmarks per hand with MediaPipe Hands and classified the normalised landmark vector with a lightweight one-dimensional convolutional neural network (1D-CNN). A dataset of 31,200 samples covering 26 letters was collected from 12 signers, and the model was compared with Random Forest, a multilayer perceptron and an image-based MobileNetV2 under signer-dependent and signer-independent protocols. The 1D-CNN achieved an accuracy of 99.2% in the signer-dependent setting and a macro F1-score of 93.7% on unseen signers, exceeding MobileNetV2 by 12.1 percentage points while using about 40 times fewer parameters. Landmark normalisation contributed the largest improvement. The pipeline ran at about 28 frames per second on a laptop CPU and reached 91.5% accuracy in a live test with five new users, showing that landmark-based recognition was an efficient and robust basis for BISINDO assistive applications.

Authors

Publication Details

Journal
Journal of Information Technology computer science and Electrical Engineering
Published
2026-11-18
DOI
https://doi.org/10.61306/4me6nh98
Primary Topic
Hand Gesture Recognition Systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Networks and MediaPipe Framework

Dimas Nugroho, Putri Maharani
Journal of Information Technology computer science and Electrical Engineering
Hand Gesture Recognition Systems
article

Real-Time Indonesian Sign Language Recognition Using Convolutional Neural Networks and MediaPipe Framework

Dimas Nugroho, Putri Maharani
article en

Abstract

Few hearing Indonesians understand Bahasa Isyarat Indonesia (BISINDO), and most existing recognition systems classify raw images, which makes them sensitive to background, lighting and signer appearance. This study developed a real-time BISINDO alphabet recognition system that extracted 21 three-dimensional landmarks per hand with MediaPipe Hands and classified the normalised landmark vector with a lightweight one-dimensional convolutional neural network (1D-CNN). A dataset of 31,200 samples covering 26 letters was collected from 12 signers, and the model was compared with Random Forest, a multilayer perceptron and an image-based MobileNetV2 under signer-dependent and signer-independent protocols. The 1D-CNN achieved an accuracy of 99.2% in the signer-dependent setting and a macro F1-score of 93.7% on unseen signers, exceeding MobileNetV2 by 12.1 percentage points while using about 40 times fewer parameters. Landmark normalisation contributed the largest improvement. The pipeline ran at about 28 frames per second on a laptop CPU and reached 91.5% accuracy in a live test with five new users, showing that landmark-based recognition was an efficient and robust basis for BISINDO assistive applications.

Journal of Information Technology computer science and Electrical Engineering
Hand Gesture Recognition Systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon