YOSO-SLAM: A Real-Time Object Visual SLAM for Dynamic Scenes with Semantic Three-Dimensional Mapping

Abstract Recent advances in mobile robotics, such as drones with onboard cameras, have made Visual Simultaneous Localization and Mapping (VSLAM) methods more practical. These methods use images from cameras, which are compact, cost-effective, and enhance data collection and inspection. However, traditional VSLAM systems are typically designed for static environments and lack semantic information for scene understanding. Current VSLAM approaches combining deep learning and geometry for dynamic scenes often increase complexity and computational load. This paper proposes YOSO-SLAM (YOLO Semantic Object SLAM), an enhanced real-time VSLAM framework that generates a semantic three-dimensional (3D) map for dynamic scenes while maintaining manageable computational demands and high pose estimation accuracy. YOSO-SLAM incorporates the ORBSLAM3 pipeline and integrates YOLOv8 for object detection and dynamic feature removal. YOLOv8, known for its fast and robust inference, outperforms methods like SegNet and Mask-R-CNN. The framework also includes an object database that stores detection results and semantic information obtained via depth filtering. The 3D map is enhanced with semantic labels and rendered using Octomap. YOSO-SLAM significantly improves accuracy compared to ORBSLAM3, reducing absolute trajectory error (ATE) by 97.92% and relative pose error (RPE) by 53.23% in a challenging sequence from the TUM RGB-D dataset. Similar improvements are observed in Octomap accuracy with the ICL-NUIM dataset. Compared to other well-known VSLAM systems, YOSO-SLAM achieved the highest accuracy in the TUM RGB-D sequence (fr3_w_rpy) and shows strong performance in other sequences as well. The semantic labeling within the Octomap also enhances scene understanding for both robots and humans.