The effects of camera perspective and augmentation on performance, situation awareness and mental workload of remote assistants of highly automated vehicles

In real-world operations, highly automated vehicles (HAVs, SAE Level 4) face many traffic situations they cannot cope with, e.g., situations with adverse weather. Remote human support may help to resolve such situations to increase robustness of HAV operations. In this task context, human-machine interfaces (HMIs) for remote operators of HAVs often present traffic situations similar to the driver's perspective. However, this first-person view is associated with shortcomings including the occlusion of relevant objects on the road or the distortion of distance and angle perception. These shortcomings may affect the performance of the remote operator. An experimental lab study with 37 participants was carried out to investigate if three different camera perspectives affect operator performance, situation awareness, and other operator-related variables in a remote assistance task at a busy urban intersection with mixed traffic. Additionally, the interplay of camera perspectives and video augmentation by visualizing additional sensor data was investigated in an environment with and without adverse weather due to fog. Results indicated that certain performance indicators including decision time were affected by camera perspective. The positive and compensatory impact of augmentation under poor visibility conditions in adverse weather was replicated. Findings suggest that the most suitable perspective highly depends on the specific scenario. The results will help design context-sensitive HMIs for remote assistance of HAVs.