Auditory, Visual, or Both? Comparing Visual and Auditory Representations of Game Elements in a Gamified Image-Tagging Task

In gamification users show increased motivation and engagement towards tasks. So far, this effect has mostly been empirically tested based on the visual depiction of game elements, while research on the use and addition of auditory aspects is sparse. In this work we investigate the effect of different modalities of game elements (auditory, visual, audiovisual) on user experience, perception and performance in a gamified image-tagging task. We approached this via an online validation survey to find suitable sound effects (N=50), a main quantitative study (N=109) and a qualitative semi-structured interview (N=9). Our results show that while visual gamification increases performance, auditory and audiovisual gamification had no effect. However, they were shown to have an influence on the user's flow state. Our qualitative follow-up study shed light on underlying reasons and revealed each modality has its own drawbacks and advantages and that combining both visual and auditory aspects was preferred by participants.