Smart Docent System for Visually Impaired People with Mobile-Based Artificial Intelligence of Things

With the increasing demand for high-quality user experiences in museums and art galleries, interest in smart guidance systems has grown. These systems aim to provide detailed information about the artwork and deliver audio explanations related to the pieces of interest. Traditional guidance systems typically use techniques such as numbering, QR codes, and beacons. However, these methods can be challenging for visually impaired individuals, as they rely on visual cues. To address this issue, this paper introduces a novel smart docent system designed to enhance the gallery experience for visually impaired visitors. The system features two primary functional frameworks, supported by Artificial Intelligence (AI) models: user escorting and audio touring. These frameworks utilize mobile-based sensing, dynamic Bluetooth Low Energy (BLE) beaconing, and edge computing. The system employs an on-device object detection model to minimize communication delays caused by large amounts of visual data. Although BLE beacons are statically installed within the art gallery, they dynamically update location IDs. Edge computing helps correct positioning errors based on the user's movement history and direction. Moreover, the system adaptively provides an audio tour based on the user's behavior, such as approaching or passing by artwork, or leaving an exhibit while listening to audio descriptions.