Improving road user perception in adverse weather: An augmented human–machine interface for remote assistants of automated vehicles

• We present an interface for displaying sensor data for vehicle remote assistance. • A user study investigated the augmented interface under fog and higher workload. • The interface reduced collisions and improved situation awareness and usability. • The interface may accelerate the safe deployment of highly automated vehicles. Highly automated vehicles (SAE level 4) occasionally require human support. A remote assistant can help the vehicle from a distance. To date, virtually all human–machine interface (HMI) concepts for remote operation rely on cameras on board the vehicle for assessing its environment. However, in adverse weather, a remote assistant may not be able to decide based on video streams only. We propose an HMI concept for augmenting video streams with visualized data from vehicle sensors. In an experiment, 34 participants assisted the driving automation in a left-turn task using the augmented HMI concept. Visibility was varied by inducing fog. Results show that the augmented HMI effectively supported participants in their remote assistance task. Particularly when foggy, the augmentation reduced collisions, improved situation awareness, and received higher usability ratings. The results imply that augmentation is effective for increasing safety, especially in poor-visibility environments. Future research should examine implications for workplace design.