SYNERGY+: A Large-Scale, Curated Benchmark Dataset of Expert Screening Decisions for Systematic Reviews

Progress in natural language processing and information retrieval increasingly depends on large-scale evaluation data grounded in authentic high-quality human judgment. Such data are most valuable in high-stakes domains where annotation requires niche expertise, and they are becoming harder to obtain as generative AI and AI-assisted annotation tools make it difficult to guarantee that newly created datasets reflect genuine human decisions rather than synthetic or model-influenced labels. Here we present SYNERGY+, a curated collection of 119 SLR datasets comprising 494,201 screened records across four research domains (Health Sciences, Social Sciences, Physical Sciences, and Life Sciences), in which every record was evaluated by at least one human domain expert. Each dataset was selected from an initial pool of 212 candidates through a rigorous quality assessment covering licensing, data completeness, and publication integrity. These SLRs and their datasets were published between 2011 and 2025, predating the use of generative AI in screening. All records are enriched with bibliographic metadata via hierarchical matching against OpenAlex. SYNERGY+ is released under a CC0 license, with abstracts included for 168,969 open-access records where redistribution was permitted. The dataset is distributed through a Python package with a predefined stratified train/test split for standardized benchmarking. SYNERGY+ represents, to our knowledge, the largest and most methodologically rigorous benchmark for AI-assisted title and abstract screening to date, supporting applications spanning technology assisted SLRs, large language model evaluation, and evidence synthesis research.