PhotoTalk: an LLM-based photo album browsing assistance with chatbot for people with visual impairments

Abstract People with visual impairments often face difficulties finding desired photos in their smartphone albums. To identify the needs and challenges in photo browsing, we first conducted an interview with 10 participants with visual impairments. Based on the findings, we propose PhotoTalk , an intelligent photo album that assists users with photo browsing tasks by making requests to an LLM-based chatbot for finding photos of specific objects ( e.g., ‘Find photos with flowers’ ), visual question answering ( e.g., ‘What color are the flowers? ’) and recommendations ( e.g., ‘Which photo looks better for my Instagram? ’). As for the evaluation, we conducted another user study with 13 participants with visual impairments, which resulted in high ratings in terms of System Usability Scale. Specifically, participants perceived PhotoTalk as easy to use and as reducing the effort required for photo finding relative to their current practices. They also noted that it could address the challenges they previously encountered ( e.g., finding photos that are not well-remembered, distinguishing between similar photos). We discuss design implications for future chatbot-based assistance to enhance visual accessibility and independent photo browsing for people with visual impairments.