Multi-task traffic scene perception algorithm based on multi-scale prompter

Abstract Traffic sceSne perception involves the interrelated tasks of 3D object detection, semantic segmentation, and depth estimation. To concurrently learn multi-scale task-general, task-specific, and cross-task complementary features, a multi-task traffic scene perception algorithm based on a visual prompter is developed. The algorithm introduces a multi-scale prompt learning method to obtain rich multi-scale feature maps and prompt words. A prompt selector module and an up-sample aggregate module are designed to process the output feature maps and prompt words effectively, aggregating the feature maps to a uniform scale. Additionally, a prompt teaching method enhances efficient multi-task learning. Experimental results on the Cityscapes-3D dataset demonstrate that the proposed algorithm outperforms current mainstream algorithms, achieving an mDS of 33.1% for 3D object detection, an mIoU of 79.9% for semantic segmentation, and an RMSE of 4.20 for depth estimation.