Road Adhesion Information Estimation of Connected Autonomous Vehicle Based on Digital Twin and Vision Sensor Fusion

The road adhesion coefficient is crucial information for connected autonomous control. However, it is challenging to obtain based solely on current vehicle state sensors. This paper proposed a novel road adhesion coefficient estimation method based on the digital twin framework and combing of vehicle state sensor and vision sensor. Utilizing the vehicle state sensor, a nonlinear observer dynamics model of tire force is established. Then, an improved Innovation Adaptive Estimation Unscented Kalman Filter (IAE-UKF) algorithm is designed for road adhesion coefficient estimation. Simultaneously a deep convolutional neural network is adopted to classify the road surfaces type based on the images from visual sensor. Multi-sensor data fusion is performed by mapping visual identification labels to reference values via a lookup table, followed by spatiotemporal synchronization with the dynamics based approach. A distributed cooperative estimation mechanism is developed to address potential failures in either estimator. Simulation and experimental results show that the proposed strategy effectively integrates a variety of sensor information based on the digital twin framework. Compared to traditional dynamics based methods, the introduction of tire dynamic characteristics discrimination significantly reduces its reliance on high tire excitation. Furthermore, the proposed estimation method can maintain the accuraccy and reliability in the situation of vision sensor misidentification.