Evaluating Gaze Event Detection Algorithms: Impacts on Machine Learning-based Classification and Psycholinguistic Statistical Modeling

Eye movements offer valuable, non-invasive insights into cognitive processes and are widely used in both psycholinguistic research and machine-learning applications, such as assessing reading comprehension and cognitive load. These applications typically rely on fixations and saccades detected through gaze event algorithms, which may be either proprietary or open-source. The impact of different gaze event detection algorithms on subsequent analysis is underexplored and often overlooked. This study investigates how two threshold-based algorithms, I-DT and I-VT, influence both machine-learning classification tasks and psycholinguistic statistical modeling. Using diverse datasets-including stationary, remote, and VR eye-tracking data across multiple sampling frequencies-our findings show significant differences in downstream performance. For ML tasks, I-DT generally outperforms I-VT, with I-VT being highly sensitive to threshold choices. In psycholinguistic analysis, results confirm established findings only when thresholds align with established fixation metrics, emphasizing the importance of appropriate threshold selection for meaningful analysis. Our code is publicly available: https://github.com/aeye-lab/eye-movement-preprocessing.