Real-Time Bidirectional Sign Language Translation Using MobileNet and Tensorflow Lite

Sign language is the most predominant means of communication for the hearing and speech-impaired population; however, with the lack of accessible translation tools, communication poses a great barrier. Most current solutions have revolved around one-way translation between signers and non-signers. This paper presents a real-time bidirectional sign language translation system designed to bridge this gap. This system utilizes the power of MobileNetV2 for gesture recognition and TensorFlow Lite for mobile optimization to provide users with smooth translations between sign language gestures and text in an entirely offline system. Another Blender-based 3D animation system will be integrated within a Flutter-based mobile application in order to develop better text-to-sign conversion processes for dynamic visual representations of gestures. The recognition system provides gesture recognition with low latency, obtaining a recognition rate of 80.5% accuracy. It makes the hearing and speech-impaired community more inclusive, accessible, and independent since it eliminates dependence on third-party interpreters and internet connectivity.