3D Multi-object Tracking Using Instance Segmentation of Stereo Camera Images in Autonomous Vehicles

This study proposes a novel approach for estimating the location of an obstacle during autonomous driving using a stereo camera. Deep learning-based object detection, YOLOv8 was employed to secure real-time performance and achieve high accuracy in detecting the obstacles. Despite the advantages of this scheme, its overall performance may be hindered by the introduction of noise due to uncertainty in IoU(Intersection over Union) and recall when estimating three-dimensional positions using bounding boxes. This can lead to serious problems in path planning or vehicle control and cause fatal accidents during autonomous driving. Therefore, a 3D object position estimation method has been devised in this work in order to resolve this issue. This technique provides robust protection against uncertainty in detecting obstacles through a synergistic effect of deep learning-based instance segmentation, feature point tracking, and multiple object tracking. Thus, a new method for estimating the 3D location of objects using instance segmentation and multiple object tracking has been developed in this work, which is available in real time for real-world applications.