DSVT: Dynamic 3D Surround View for Tractor-Trailer Vehicles Based on Real-Time Pose Estimation with Drop Model

In recent years, 3D surround view systems have attracted a lot of attention in the field of advanced driver assistance systems (ADAS). However, the foundational assumption of unchanging camera poses in traditional 3D surround view systems, which is designed for single-unit vehicles, results in a failure to manage the non-rigid connections characteristic of tractor-trailer vehicles. Moreover, tractor-trailer vehicles have the feature of long bodies and large wheelbases, leading to severe distortions and abrupt changes in the rendering results of previous 3D texture mapping models. In this paper, we propose DSVT, a dynamic 3D surround view system for tractor-trailer vehicles, designed to address the aforementioned issues. Specifically, we develop a dynamic surround image stitching algorithm based on relative pose estimation, which estimates the relative poses between cameras and stitches all images together to generate a 2D panoramic image. Subsequently, a novel 3D drop model is proposed, mapping the 2D panoramic image onto the 3D model for panoramic viewing. Our system can run in real time on Nvidia AGX Orin. Experimental results in real tractor-trailer scenes show that our system can achieve more accurate and natural visual effects.