CiThruS2: Open-Source Simulation Framework for Vision-Based ADAS Development

Vision-based Advanced Driver Assistance Systems (ADAS) rely on cameras and other sensor modalities whose correct functionality tends to be guaranteed through rigorous physical testing. However, the increasing complexity of ADAS makes exhaustive real-world testing impractical, so the automotive sector is increasingly turning to more agile virtual test environments. This paper presents the latest version of our open-source CiThruS2 traffic simulation framework that introduces a flexible vision pipeline system for seamlessly transitioning between virtual and physical worlds. The system supports visual data streaming in the High Efficiency Video Coding (HEVC) format over the Real-time Transport Protocol (RTP), thereby offering an efficient and standardized communication interface for vision-based ADAS testing and validation. The pipeline usage is illustrated with a diminished reality (DR) use case, where depth image-based rendering is applied to virtually eliminate blind spots in the driver's field of view and raise their awareness of occluded road users.