Assessing the effect of arousal on performance in a virtual reality narrative scenario using biological signals

When analysing a user’s experience of virtual reality (VR), factors such as their level of technical familiarity, proficiency with immersive technology and concomitant degree of physiological arousal inside a VR experience can have a significant impact on their performance, sense of presence and engagement. We have designed a modular narrative system to manipulate a user’s levels of arousal in order to keep them within an optimal range for performance, which we hypothesise to be between not too stressed (high arousal) and not too bored (low arousal). We do so by instantiating an increasing number of simultaneous tests and environmental changes at different points during a VR experience. Changes in autonomic signals - such as heart rate, heart rate variability, galvanic skin response, and skin temperature - reveal changes in the levels of participant arousal. The user is embodied in a gender-specific out-group (Muslim) avatar that is subjected to an increasingly stressful event (a series of verbal Islamophobic attacks from a non-player character). We measure performance in a series of simultaneous multiple choice listening comprehension tasks (averaged to create a “narrative task score”) undertaken as the scene unfolds, and a post-treatment recall task. As a pilot experiment, our primary objective is to validate the effectiveness of the system as a means of stress manipulation and thereby assess the impact and correlation that different levels of arousal have on task performance and biological signals. Results revealed a statistically significant difference in narrative task performance between stress levels, confirmed by a one-way ANOVA ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m1"><mml:mrow><mml:mi>F</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>2,45</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mn>5.06</mml:mn></mml:mrow></mml:math> , <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m2"><mml:mrow><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mn>0.02</mml:mn></mml:mrow></mml:math> , <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m3"><mml:mrow><mml:mi>S</mml:mi><mml:mi>E</mml:mi><mml:mo>=</mml:mo><mml:mn>23.89</mml:mn></mml:mrow></mml:math> ). The low stress group achieved the highest mean VR score ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m4"><mml:mrow><mml:mi>M</mml:mi><mml:mo>=</mml:mo><mml:mn>73.12</mml:mn></mml:mrow></mml:math> , <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m5"><mml:mrow><mml:mi>S</mml:mi><mml:mi>D</mml:mi><mml:mo>=</mml:mo><mml:mn>15.96</mml:mn></mml:mrow></mml:math> ), followed by the high ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m6"><mml:mrow><mml:mi>M</mml:mi><mml:mo>=</mml:mo><mml:mn>63.25</mml:mn></mml:mrow></mml:math> , <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m7"><mml:mrow><mml:mi>S</mml:mi><mml:mi>D</mml:mi><mml:mo>=</mml:mo><mml:mn>18.23</mml:mn></mml:mrow></mml:math> ) and medium stress groups ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m8"><mml:mrow><mml:mi>M</mml:mi><mml:mo>=</mml:mo><mml:mn>51.81</mml:mn></mml:mrow></mml:math> , <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m9"><mml:mrow><mml:mi>S</mml:mi><mml:mi>D</mml:mi><mml:mo>=</mml:mo><mml:mn>23.66</mml:mn></mml:mrow></mml:math> ). Our hypothesis that the medium stress condition would produce the best performance was therefore rejected. Comparing heart rate variability (HRV) metrics, the Stress Index showed a statistically significant difference between conditions ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m10"><mml:mrow><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mn>0.043</mml:mn></mml:mrow></mml:math> , with significant within-condition changes also observed in the LF/HF ratio ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m11"><mml:mrow><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mn>0.005</mml:mn></mml:mrow></mml:math> in low stress and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m12"><mml:mrow><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mn>0.008</mml:mn></mml:mrow></mml:math> in high stress), further demonstrating the physiological changes between stress levels.