Effects of Confidence Visualization in Automated Assembly Instructions

Automatic Augmented Reality (AR)-based work support systems seamlessly display the next step instruction when the current task step completion is detected, all without requiring explicit user interaction. However, whenever the progress estimation fails it can lead to confusion and loss of trust in the system. To support the user’s understanding of the reliability of the displayed instructions, we display the confidence of the current step. To explore the effects the presented confidence has on user interaction with the system, we conducted a Wizard-of-Oz style user study with 12 participants. Participants had to assemble a LEGO model with and without confidence visualization, where the confidence for each step was predefined. Our results showed that while the visualization did not increase the user’s performance, participants preferred having the confidence displayed to them, especially when it was low. Our findings provide insights into the design of enhancing the user’s perception of uncertainty in current step detection in automated instruction systems.