Developing and validating an adaptive multi-layer vehicle trajectory reconstruction method for outlier removal

• Proposes an adaptive multi-layer vehicle trajectory reconstruction method. • Assess the closeness between the ground truth and the reconstructed trajectories. • A window size adaptive to the varying lengths of the detected outlier sequences. • An objective reconstruction mechanism by leveraging information of surrounding vehicles. • Guaranteed consistency and its performance is rigorously assessed and compared. Trajectory data is vital for traffic flow studies, and aerial photography-based methods are increasingly used to collect such data. However, these datasets often contain errors from various sources, which can be exacerbated by numerical derivative processes. Previous efforts have not fully addressed some of these issues such as consistency, varying length of outlier sequences, and unknown ground truth trends. Moreover, existing validation methods are often indirect and problematic. To address these limitations, we propose an adaptive multi-layer vehicle trajectory reconstruction method, which consists of three modules: the Initial Window Arrangement module to ensure the precise alignment between the reconstruction window and detected outlier fragments, maintaining internal consistency at boundary points; the Window Size Feasibility Test module to adaptively determine the window size according to varying length of outlier sequences, and XGBoost-based Ground Truth Estimation module to be combined with a least-square-based objective function to significantly improve reconstruction accuracy and more closely replicate the underlying trend. Additionally, we introduce the jerk-based reconstruction, which outperforms the acceleration-based reconstruction. To reliably assess and select the best ground truth estimation scheme and objective function, a novel synthetic dataset containing both the ground truth and realistic outlier fragments is proposed. Subsequently, a comparative evaluation of six different outlier removal methods was conducted using Zen Traffic Data. The validation results, utilizing both the synthetic dataset and Zen Traffic Data, demonstrate the exceptional performance of the proposed method across various evaluation perspectives. The good performance of the proposed outlier removal method is further demonstrated by comparing the IDM calibration results using the trajectories with and without outliers being removed. With just one parameter (jerk anomaly threshold), the parameter settings of our method are more objective and generalizable.