Configurable gestures in VR: building flexible metaverse systems for special needs education

This study explores the integration of configurable word-related iconic gestures in virtual reality (VR) learning systems, aiming to enhance learner engagement and attention while maintaining pedagogical flow. While VR offers immersive experiences, adapting content efficiently to diverse learner needs remains challenging. We developed a system in which gestures can be toggled on/off as a redundancy layer (gesture+speech) without changing pathway structure, speech timing, or assessment checkpoints. Using the Soqqle Classlet platform, we implemented this configuration-driven architecture and evaluated it with 23 plurilingual participants in special needs education during a translanguaging communication pathway. Enabling gestures produced a more concentrated gaze distribution across in-scene targets: Gini was higher (paired t-test, p = 0.0074) and HHI was higher (p = 0.0398), while entropy did not differ between conditions. Task performance and interaction logs were broadly comparable between conditions. Notably, the newly introduced attentional allocation index (IDX) related to self-reported experience factors (e.g., affect and intention), highlighting that configurability can shift process-level attention even when outcomes remain stable. These findings suggest that gestures can focus attention without necessarily improving performance, supporting configurable VR deployments that balance flexibility with instructional effectiveness for SEN contexts.