Effect of Display Response Time on Sense of Agency and Brain Activity During Human–machine Interface Device Operation

This study investigated the effect of display response delays, i.e., the time lag between operation input and display output, on human–machine interface (HMI) device operation, emphasizing commander-type devices, e.g., those used for in-vehicle infotainment. While operating the HMI device, simultaneous processing of visual and tactile information, e.g., operating the device while gazing at the display, is required. This investigation was conducted to clarify how varying response delays in visual feedback after tactile feedback with device operation affect usability in terms of various aspects, e.g., task efficiency, subjective workload, and the sense of agency (SoA). This study employed a targeting task, in which the participants engaged in HMI device operation under a delay time ranging from 0–120 ms. The subjective workload was evaluated using NASA Task Load Index (NASA-TLX) scale, and the SoA was assessed quantitatively through Near-infrared Spectroscopy (NIRS) measurements of cerebral hemodynamics, which provide insights into the user’s psychological and physiological responses to delay. The findings revealed a nuanced relationship between response delays and usability during HMI device operation. We found that slight delays (40– 80 ms) do not necessarily detract from user workload or efficiency. Instead, the slight response delays can enhance the user’s SoA, which suggests that there is an optimal delay window that balances immediate feedback with the cognitive and sensory processing times inherent to human perception. These findings suggest that incorporating slight delays within the software framework of HMI system design effectively enhances the SoA, thereby enabling more intuitive user interactions and improving usability.