Braille Character Recognition Independent of Lighting Direction Using Object Detection and Segmentation Models

Braille text is used worldwide as a means of communication for the visually impaired. Although braille is primarily used by visually impaired individuals, sighted individuals may also want to read braille documents. For example, teachers at schools for the visually impaired may need to check homework or assignments written in braille, or care workers supporting the daily lives of visually impaired people may need to read braille documents. To meet such social demands, optical braille recognition technologies have been developed, typically using cameras and scanners. In particular, methods using deep learning models have achieved high accuracy in recent years. Because braille is represented by embossed dots on thick paper, the appearance of shadows varies significantly depending on the lighting angle. Therefore, the angle and intensity of light significantly affect the accuracy of braille recognition. Specifically, in the case of double-sided braille, where braille is embossed on both sides, distinguishing between raised and recessed dots is necessary. Previous studies have mostly imposed constraints on the lighting angle, such as limiting the imaging method to scanners or requiring the light source to be positioned on top of the braille document when using a camera. However, for practical use in daily life, it is preferable to recognize braille documents independent of the lighting direction. In this study, we created a dataset of braille images captured under various angle of lighting and developed a braille recognition model that is independent of the lighting angle by fine-tuning the object recognition and segmentation models. We implemented and compared RetinaNet, which was used in previous research, with the anchor-free YOLOX model as an object detection model. We also implemented BraU-Net, a customized segmentation model based on UNet, and compared it with object detection models. The object detection model achieved an accuracy of mAP50=0.98 or higher, regardless of the lighting angle.