Tri-manual interaction in hybrid BCI-VR systems: integrating gaze, EEG control for enhanced 3D object manipulation
Brain-computer interface (BCI) integration with virtual reality (VR) has progressed from single-limb control to multi-limb coordination, yet achieving intuitive tri-manual operation remains challenging. This study presents a consumer-grade hybrid BCI-VR framework enabling simultaneous control of two biological hands and a virtual third limb through integration of Tobii eye-tracking, NeuroSky single-channel EEG, and non-haptic controllers. The system employs e-Sense attention thresholds (>80% for 300 ms) to trigger virtual hand activation combined with gaze-driven targeting within 45° visual cones. A soft maximum weighted arbitration algorithm resolves spatiotemporal conflicts between manual and virtual inputs with 92.4% success rate. Experimental validation with eight participants across 160 trials demonstrated 87.5% virtual hand success rate and 41% spatial error reduction (<i>σ</i> = 0.23 mm vs. 0.39 mm) compared to traditional dual-hand control. The framework achieved 320 ms activation latency and 22% NASA-TLX workload reduction through adaptive cognitive load management. Time-frequency analysis revealed characteristic beta-band (15-20 Hz) energy modulations during successful virtual limb control, providing neurophysiological evidence for attention-mediated supernumerary limb embodiment. These findings demonstrate that sophisticated algorithmic approaches can compensate for consumer-grade hardware limitations, enabling laboratory-grade precision in accessible tri-manual VR applications for rehabilitation, training, and assistive technologies.
Authors
- S. R. Lee
- Jian Teng (ORCID: https://orcid.org/0000-0003-1876-989X)
- Sukyoung Cho
Institutions
- Lingnan Normal University (CN)
- Sehan University (KR)
- Zhuhai Institute of Advanced Technology (CN)
Publication Details
- Journal
- Frontiers in Neurorobotics
- Published
- 2025-08-14
- DOI
- https://doi.org/10.3389/fnbot.2025.1628968
- Citations
- 1
- Primary Topic
- Gaze Tracking and Assistive Technology
- Type
- article
- Field-Weighted Citation Impact
- 1.94