How Can I Trust You? The Effect of Risk and Automation Failures on Trust and Reliance Behavior

ObjectiveWe examine how risk and automation failures in conditional driving automation (SAE Level 3) influence drivers' calibration of trust and reliance behavior in the form of system use and monitoring.BackgroundConditionally automated driving brings a challenging new role for drivers, who are permitted to engage in non-driving-related activities but must take back control in certain situations.MethodsParticipants completed three drives in a driving simulation with conditional driving automation. The first drive was with low risk and the second drive was with high risk implemented in the simulation. The third drive included either early or late automation failure.ResultsParticipants reported lower trust, took over manual control more often, and monitored more when driving under high risk than when driving under low risk. After experiencing an automation failure, trust decreased immediately but fully recovered over time. Driver's monitoring increased and decreased immediately as the failure started and ended. The timing of automation failure did not influence its impact on trust.ConclusionThe results indicate that drivers respond appropriately to risk. Trust develops dynamically in case of an automation failure, but failure timing does not influence this process. From an applied perspective, drivers would benefit from assistance in re-calibrating trust after automation failure.ApplicationBased on our findings, we argue that incorporating drivers' mental model formation process into the feedback loop of trust and reliance behavior calibration could enhance the theoretical understanding of trust calibration.