A Custom Transformer-Based Framework for Joint Traffic Flow and Speed Prediction in Autonomous Driving Contexts

Short-term traffic prediction is vital for intelligent transportation systems, enabling adaptive congestion control, real-time signal management, and dynamic route planning for autonomous vehicles (AVs). This study introduces a custom Transformer-based deep learning framework for joint forecasting of traffic flow and vehicle speed, leveraging handcrafted positional encoding and stacked multi-head attention layers to model multivariate traffic patterns. Evaluated against baselines including Long Short-Term Memory (LSTM), Support Vector Machine (SVM), Random Tree, and Random Forest on the Next-Generation Simulation (NGSIM) dataset, the model achieves 94.2% accuracy (Root Mean Squared Error (RMSE) 0.16) for flow and 92.1% accuracy for speed, outperforming traditional and deep learning approaches. A hybrid evaluation metric, integrating RMSE and threshold-based accuracy tailored to AV operational needs, enhances its practical relevance. With its parallel processing capability, this framework offers a scalable, real-time solution, advancing AV ecosystems and smart mobility infrastructure.