Evaluating the Efficacy of Smart Saliency Detection System for Visual Prosthesis Users: An Experimental Comparison Across Various Visual Prosthesis Implants

BACKGROUND: Individuals with visual prostheses often struggle to locate specific objects due to limited visual input. Current systems process visual information but fail to effectively highlight or prioritize objects based on user needs. METHODS: This study investigates a user-centric object highlighting system designed to assist retinal prosthesis users in finding misplaced objects in a controlled experimental setting. The system uses pre-trained multimodal models, allowing users to specify objects through spoken instructions, which are then highlighted in the camera frame. We evaluated the system's performance in a simulated environment with 18 sighted participants acting as virtual patients. We examined how various retinal implant resolutions and stimulation points, from low to high (60-1600 electrodes), impact object recognition. RESULTS: Mixed-effects logistic regression, both frequentist and Bayesian, revealed significant variance in recognition outcomes based on fixed and random effects. High-resolution implants achieved the best and most consistent recognition rates, while lower-resolution implants showed suboptimal recognition. Notably, across all implants, larger objects were recognized more effectively compared to smaller items, indicating that even higher-resolution implants struggle with smaller objects. The characteristics of these objects, particularly size and distinct features, played a crucial role in their recognition performance. This underscores the necessity for effective detection systems tailored to the capabilities of implants, especially those with lower resolution, which lack sufficient detail for independent object recognition. CONCLUSION: These findings provide valuable insights for enhancing user-centric object highlighting systems and inform the development of real-world testing in complex environments.