Driver's Hand Trajectory-Guided Takeover Intention Inference in Conditionally Automated Driving

Human intervention is critical in situations where the autonomous system issues a takeover request in conditionally driving. Effective human-machine collaboration and safe transition require the timely and accurate identification of the driver's takeover intentions. Given the challenges associated with the opacity of decision-making processes in standalone end-to-end models for inferring takeover intentions, this paper introduces an approach for inferring takeover intention based on driver-hand trajectory estimation, which leverages the physical and observable actions as an interpretable indicator. We first presented a comprehensive framework designed to acquire preliminary data on the driver's hand and steering wheel position. This framework utilized MediaPipe for the acquisition of hand coordinates, EfficientDet for steering wheel localization, and MiDaS for depth estimation. Then, the Extended Kalman Filter (EKF) model was integrated into the Gated Recurrent Unit (GRU) network (GRU-EKF) to estimate hand movements and subsequently predict takeover intention combining current-time trajectory with estimated future states. The approach was validated through simulated experiments of human-machine driving interaction. In addition, a further examination is conducted to analyze the influence of six distinct combinations of historical and estimation horizons on model performance. Results show that the GRU-EKF methodology outperforms other models in terms of overall performance metrics. Using shorter estimation horizons can lead to higher accuracy and precision, whereas increasing the estimation horizon to 3 s notably improves the model's ability to correctly classify different takeover intentions. The findings can offer valuable perspectives for the design and enhancement of takeover process functionalities of Advanced Driver-Assistance Systems.