Exploring cross-modal pitch-colour associations for music learning in immersive Virtual Reality
Cross-modal mappings between pitch and colour offer promising avenues for enhancing perceptual understanding in music-related learning. Virtual Reality (VR) provides a unique platform for exploring such associations and their potential in learning musical theory. This study explores the use of pitch-colour associations within an immersive VR music learning environment. Specifically, it examines how pre-defined pitch-colour mappings (those where colours are already mapped to a specific note) and personalised mappings (those where participants choose which colours are mapped to a specific note) can support music learning tasks in VR. Thirty participants were divided into two groups—one using pre-defined mappings and the other creating personalised ones. Each group completed eight music lessons based on the C Major Scale using a VR headset. The between-subjects design examined whether personalised mappings support increased task performance, users’ perceived learning and user engagement, compared to the pre-defined group, using both quantitative measures (error rates and task completion time) and qualitative template analysis. No statistically significant performance differences were found between groups, although the small sample size may have limited the power of the quantitative analysis. In contrast, the qualitative data provided valuable insights into participants’ experiences, highlighting how both mapping strategies shaped engagement and understanding with the music lessons. Notably, significant improvements in error scores and completion times were observed across the eight music lessons for both groups. Additionally, VR design elements—including locomotion, interaction, and multimodal cues—were found to influence immersion and usability. These findings demonstrate the potential of VR applications that use a pitch-colour cross-modal mapping design to support music-related tasks such as interval recognition, pitch differentiation, and ear training. They also inform future design considerations for cross-modal interaction and its potential in immersive learning environments.
Authors
- Laura Maye (ORCID: https://orcid.org/0000-0003-2711-887X)
- Maurizio Mancini (ORCID: https://orcid.org/0000-0002-9933-8583)
- Patrick J. O’Toole (ORCID: https://orcid.org/0000-0001-6130-1024)
- Ian J. Pitt (ORCID: https://orcid.org/0000-0001-5207-2506)
Institutions
- University College Cork (IE)
- Sapienza University of Rome (IT)
Publication Details
- Journal
- Computers & Education X Reality
- Published
- 2026-11-21
- DOI
- https://doi.org/10.1016/j.cexr.2026.100191
- Primary Topic
- Multisensory perception and integration
- Type
- article
- Field-Weighted Citation Impact
- 0.00