Refining the Performance of Neural Networks with Simple Architectures for Indonesian Sign Language System (SIBI) Letter Recognition Using Keypoint Detection

The diversity of non-verbal communication styles among persons with disabilities in Indonesia highlights the urgent need for technological solutions that support accessibility in both workplace settings and social contexts. This study proposes a novel approach to improving neural network performance through the use of simple architectures for recognizing Indonesian Sign Language (SIBI) letters M and N, by applying keypoint detection while accounting for hand size variations (17–22 cm). Four models were evaluated: YOLOv5 based on image detection, as well as VGG-16, Attention, and Multi-Layer Perceptron (MLP) developed using keypoint detection. The evaluation was conducted in real-time, taking into account accessories such as rings, watches, and gloves, as well as varying lighting intensities to simulate real-world user environments. The novelty lies in the integration of keypoint detection into lightweight architectures, which significantly improves accuracy and resilience against visual disturbances (noise). The MLP model achieved the best performance, with an accuracy of 94% for M and 93% for N, outperforming more complex approaches such as YOLOv5, which showed a significant drop in accuracy under disturbed conditions. The integration of VGG-16 with Attention resulted in underfitting, emphasizing that complexity does not always correlate with effectiveness. These findings underscore the potential of lightweight models to enhance technological accessibility for the disabled community across various social and professional domains.