Towards Affective Sensory Substitution

Sensory substitution technologies have emerged as innovative solutions to assist individuals with sensory impairments by providing alternative modalities for perceiving the environment. However, despite their potential, these devices often face challenges related to user acceptance and effectiveness. One crucial aspect influencing the adoption of sensory substitution devices is the emotional response elicited by these technologies. Emotional engagement plays a significant role in crossmodal perception, enhancing the salience and relevance of sensory information. This paper explores the impact of emotional engagement on the perception and acceptability of sensory substitution devices, proposing the integration of emotional components to bridge the gap between crossmodal perception and user satisfaction. We present a comprehensive investigation into the role of emotions in sensory substitution, culminating in the development of a generative model capable of translating visual information into emotionally resonant auditory experiences. By addressing the emotional aspects of sensory substitution, we aim to enhance user experience and pave the way for more successful implementation and adoption of these technologies.