Code Reproducibility Check: Integrating Attention Tracking (Eye/Head/Motion) and LLMs in VR Tutorial

Virtual environments offer ecologically valid experimental settings while maintaining experimental control and supporting generalizability. This tutorial presents a Unity-based system that integrates attention tracking with large language models (LLMs) to generate contextual descriptions of attended objects in extended reality (XR) environments. To provide a practical, reproducible workflow integrating desktop-based simulated attention tracking, immersive head and controller tracking, and hardware-based eye tracking to deliver AI-generated contextual feedback. This tutorial includes implementation scripts, platform configuration guidance, a demonstration video, troubleshooting information, and project materials. This novel method automates the extraction of visual and contextual information from XR environments in real time, reducing manual annotation workload and enabling more efficient and scalable data generation.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-29
DOI
https://doi.org/10.5281/zenodo.23764113
Primary Topic
Gaze Tracking and Assistive Technology
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Code Reproducibility Check: Integrating Attention Tracking (Eye/Head/Motion) and LLMs in VR Tutorial

Funda Yildirim, Yue Li, Ali Yağız Canıgüroğlu
4 links
Zenodo (CERN European Organization for Nuclear Research)
Gaze Tracking and Assistive Technology
article

Code Reproducibility Check: Integrating Attention Tracking (Eye/Head/Motion) and LLMs in VR Tutorial

Funda Yildirim, Yue Li, Ali Yağız Canıgüroğlu
article en

Abstract

Virtual environments offer ecologically valid experimental settings while maintaining experimental control and supporting generalizability. This tutorial presents a Unity-based system that integrates attention tracking with large language models (LLMs) to generate contextual descriptions of attended objects in extended reality (XR) environments. To provide a practical, reproducible workflow integrating desktop-based simulated attention tracking, immersive head and controller tracking, and hardware-based eye tracking to deliver AI-generated contextual feedback. This tutorial includes implementation scripts, platform configuration guidance, a demonstration video, troubleshooting information, and project materials. This novel method automates the extraction of visual and contextual information from XR environments in real time, reducing manual annotation workload and enabling more efficient and scalable data generation.

Zenodo (CERN European Organization for Nuclear Research)
University of Twente (NL)
Openalex Percentile: Top 9%
Gaze Tracking and Assistive Technology
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon