Exploring the Potential of Generative AI in Song-Signing

This study explores the potential of generative AI technology to support song-signing creation and proposes a design for an AI-based song-signing authoring tool. We identified the creative process, strategies, challenges, and requirements for an AI-assisted tool through focus group interviews with two song-signing creators. The findings suggest that generative AI can be applied to automate and streamline various aspects of the song-signing creation process, including music analysis, lyrics translation, and choreography generation. Based on these insights, we propose a design of an AI-based song-signing authoring tool incorporating large language models for sign language translation and recommendation, generative dance AI for sign language visualization, and customizable sign language avatars. The development of such tools has the potential to expand the participation of d/Deaf individuals in cultural and artistic activities and contribute to an inclusive cultural and creative ecosystem.