A vision transformer-based hybrid neural architecture for automated handwritten Bangla character recognition and braille conversion

The rapid advancement of technology has led to notable changes in the current educational system. Nevertheless, there are still relatively few assisting aids that can help in teaching individuals with disabilities, such as those who are blind or visually impaired. An effective teaching strategy for those who are blind or visually impaired is braille. Although it has been digitized to produce an electronic version, handwritten characters are not considered in those versions. Studies on English character recognition have shown high accuracy, which is not the case with Bangla character recognition. We present an automated system that converts handwritten Bangla characters to braille using novel hybrid deep neural network architectures. Our approach begins with a Character Quality Assessment Framework (CQAF), which employs adaptive thresholds and comprehensive quality metrics designed explicitly for Bangla script characteristics. Building upon this foundation, we present two architectures. HybridNet-L represents our initial multi-stream design, while HybridNet-S is a redesigned lightweight variant that reduces parameters and achieves superior accuracy, making it the primary contribution of this work. To complete the system, we implement a comprehensive accessibility solution featuring real-time braille hardware interface and text-to-speech capabilities. The model effectively processes all 84 Bangla character classes including vowels, consonants, numerics, and compound characters. Extensive evaluation against seven baseline models demonstrates that our HybridNet-S achieves superior performance with 95.80% validation accuracy while maintaining computational efficiency suitable for embedded deployment. Statistical validation and ablation studies confirm the robustness and effectiveness of our multi-stream architecture for practical assistive technology applications.