Tri-manual interaction in hybrid BCI-VR systems: integrating gaze, EEG control for enhanced 3D object manipulation

Brain-computer interface (BCI) integration with virtual reality (VR) has progressed from single-limb control to multi-limb coordination, yet achieving intuitive tri-manual operation remains challenging. This study presents a consumer-grade hybrid BCI-VR framework enabling simultaneous control of two biological hands and a virtual third limb through integration of Tobii eye-tracking, NeuroSky single-channel EEG, and non-haptic controllers. The system employs e-Sense attention thresholds (>80% for 300 ms) to trigger virtual hand activation combined with gaze-driven targeting within 45° visual cones. A soft maximum weighted arbitration algorithm resolves spatiotemporal conflicts between manual and virtual inputs with 92.4% success rate. Experimental validation with eight participants across 160 trials demonstrated 87.5% virtual hand success rate and 41% spatial error reduction (<i>σ</i> = 0.23 mm vs. 0.39 mm) compared to traditional dual-hand control. The framework achieved 320 ms activation latency and 22% NASA-TLX workload reduction through adaptive cognitive load management. Time-frequency analysis revealed characteristic beta-band (15-20 Hz) energy modulations during successful virtual limb control, providing neurophysiological evidence for attention-mediated supernumerary limb embodiment. These findings demonstrate that sophisticated algorithmic approaches can compensate for consumer-grade hardware limitations, enabling laboratory-grade precision in accessible tri-manual VR applications for rehabilitation, training, and assistive technologies.

Authors

Institutions

Publication Details

Journal
Frontiers in Neurorobotics
Published
2025-08-14
DOI
https://doi.org/10.3389/fnbot.2025.1628968
Citations
1
Primary Topic
Gaze Tracking and Assistive Technology
Type
article
Field-Weighted Citation Impact
1.94
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Tri-manual interaction in hybrid BCI-VR systems: integrating gaze, EEG control for enhanced 3D object manipulation

S. R. Lee, Jian Teng, Sukyoung Cho
1 citations
Frontiers in Neurorobotics
Gaze Tracking and Assistive Technology
1.94
article

Tri-manual interaction in hybrid BCI-VR systems: integrating gaze, EEG control for enhanced 3D object manipulation

S. R. Lee, Jian Teng, Sukyoung Cho
article en
1 citations

Abstract

Brain-computer interface (BCI) integration with virtual reality (VR) has progressed from single-limb control to multi-limb coordination, yet achieving intuitive tri-manual operation remains challenging. This study presents a consumer-grade hybrid BCI-VR framework enabling simultaneous control of two biological hands and a virtual third limb through integration of Tobii eye-tracking, NeuroSky single-channel EEG, and non-haptic controllers. The system employs e-Sense attention thresholds (>80% for 300 ms) to trigger virtual hand activation combined with gaze-driven targeting within 45° visual cones. A soft maximum weighted arbitration algorithm resolves spatiotemporal conflicts between manual and virtual inputs with 92.4% success rate. Experimental validation with eight participants across 160 trials demonstrated 87.5% virtual hand success rate and 41% spatial error reduction (<i>σ</i> = 0.23 mm vs. 0.39 mm) compared to traditional dual-hand control. The framework achieved 320 ms activation latency and 22% NASA-TLX workload reduction through adaptive cognitive load management. Time-frequency analysis revealed characteristic beta-band (15-20 Hz) energy modulations during successful virtual limb control, providing neurophysiological evidence for attention-mediated supernumerary limb embodiment. These findings demonstrate that sophisticated algorithmic approaches can compensate for consumer-grade hardware limitations, enabling laboratory-grade precision in accessible tri-manual VR applications for rehabilitation, training, and assistive technologies.

Frontiers in NeuroroboticsVol. 19
Lingnan Normal University (CN), Sehan University (KR), Zhuhai Institute of Advanced Technology (CN)
Openalex Percentile: Top 13%
Gaze Tracking and Assistive Technology
1.94
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Tri-manual interaction in hybrid BCI-VR systems: integrating gaze, EEG control for enhanced 3D object manipulation — S. R. Lee, Jian Teng, et al. · Frontiers in Neurorobotics (2025) | TGRS Research Map | TGRS