S2R-Bench: A Sim-to-Real Evaluation Benchmark for Autonomous Driving

Safety is a long-standing and final pursuit in the development of autonomous driving systems, with many safety challenge arising from perception. How to effectively evaluate the safety and reliability of perception algorithms is becoming an emerging issue. Despite its critical importance, existing perception methods exhibit a limitation in their robustness, primarily due to benchmarks being entirely simulated, which fail to align predicted results with actual outcomes, particularly under extreme weather conditions and sensor anomalies that are prevalent in real-world scenarios. To fill this gap, in this study, we propose a Sim-to-Real Evaluation Benchmark for autonomous driving (S2R-Bench). We collect diverse sensor anomaly data across various road conditions to evaluate the robustness of perception methods comprehensive and realistic manner. This is the first corruption robustness dataset based on real-world scenarios, encompassing various road conditions, weather conditions, lighting intensities, and time periods. By comparing real-world data with simulated data, we demonstrate the reliability the collected data for real-world use and aim to foster research on more robust perception models for autonomous driving.

Authors

Institutions

Publication Details

Journal
Scientific Data
Published
2025-12-04
DOI
https://doi.org/10.1038/s41597-025-06255-3
Citations
1
Primary Topic
Autonomous Vehicle Technology and Safety
Type
article
Field-Weighted Citation Impact
0.68
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

S2R-Bench: A Sim-to-Real Evaluation Benchmark for Autonomous Driving

Jun Gao, Guangqi Yang, Li Wang, Huaping Liu et al.
1 citations
Scientific Data
Autonomous Vehicle Technology and Safety
0.68
article

S2R-Bench: A Sim-to-Real Evaluation Benchmark for Autonomous Driving

Jun Gao, Guangqi Yang, Li Wang, Huaping Liu, Qingshan Yang, Weida Wang, Ziying Song, Lei Yang, Jun Li, Ying Chen, Lin Liu, Liangliang Wang, Zhiwei Li, Chao Yang, Xinyu Zhang, Bin Xu, Wenhao Yu
article en
1 citations

Abstract

Safety is a long-standing and final pursuit in the development of autonomous driving systems, with many safety challenge arising from perception. How to effectively evaluate the safety and reliability of perception algorithms is becoming an emerging issue. Despite its critical importance, existing perception methods exhibit a limitation in their robustness, primarily due to benchmarks being entirely simulated, which fail to align predicted results with actual outcomes, particularly under extreme weather conditions and sensor anomalies that are prevalent in real-world scenarios. To fill this gap, in this study, we propose a Sim-to-Real Evaluation Benchmark for autonomous driving (S2R-Bench). We collect diverse sensor anomaly data across various road conditions to evaluate the robustness of perception methods comprehensive and realistic manner. This is the first corruption robustness dataset based on real-world scenarios, encompassing various road conditions, weather conditions, lighting intensities, and time periods. By comparing real-world data with simulated data, we demonstrate the reliability the collected data for real-world use and aim to foster research on more robust perception models for autonomous driving.

Scientific DataVol. 12(1)
Beijing Institute of Technology (CN), National University of Singapore (SG), Nanyang Technological University (SG), Jilin University (CN), Wuhan University of Technology (CN), Beijing Jiaotong University (CN), Jingdong (China) (CN), Suzhou Research Institute (CN), Jilin Province Science and Technology Department (CN), Chongqing University of Technology (CN), Beijing University of Chemical Technology (CN), Tsinghua University (CN)
Openalex Percentile: Top 23%
Autonomous Vehicle Technology and Safety
0.68
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.