Code Reproducibility Check: Integrating Attention Tracking (Eye/Head/Motion) and LLMs in VR Tutorial
Virtual environments offer ecologically valid experimental settings while maintaining experimental control and supporting generalizability. This tutorial presents a Unity-based system that integrates attention tracking with large language models (LLMs) to generate contextual descriptions of attended objects in extended reality (XR) environments. To provide a practical, reproducible workflow integrating desktop-based simulated attention tracking, immersive head and controller tracking, and hardware-based eye tracking to deliver AI-generated contextual feedback. This tutorial includes implementation scripts, platform configuration guidance, a demonstration video, troubleshooting information, and project materials. This novel method automates the extraction of visual and contextual information from XR environments in real time, reducing manual annotation workload and enabling more efficient and scalable data generation.
Authors
- Funda Yildirim (ORCID: https://orcid.org/0000-0003-4310-809X)
- Yue Li
- Ali Yağız Canıgüroğlu
Institutions
- University of Twente (NL)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-29
- DOI
- https://doi.org/10.5281/zenodo.23764113
- Primary Topic
- Gaze Tracking and Assistive Technology
- Type
- article
- Field-Weighted Citation Impact
- 0.00