C²RBench: A Chinese Complex Reasoning Benchmark for Large Language Models

Large language models (LLMs) have achieved remarkable progress in autonomous reasoning, evolving from basic text processing to sophisticated multimodal reasoning, a critical capability for general-purpose AI assistants.However, existing benchmarks usually fail to adequately capture the intricate multi-step reasoning demands inherent in real-world scenarios.To bridge this gap, we propose C 2 RBench: a Chinese Complex Reasoning Benchmark for evaluating multi-step, multimodal advanced reasoning capability of LLMs.C 2 RBench comprises 1,115 carefully curated Chinese tasks, which are organized into eight domain-specific subsets, each meticulously designed to mirror real-world challenges.This hierarchical benchmark features three difficulty tiers based on the number of reasoning steps required (average 8.44 steps per task), significantly exceeding existing benchmarks in cognitive complexity.Extensive evaluations of 20 LLMs (including DeepSeek-R1) and 24 multimodal large language models (MLLMs) on C 2 RBench reveal critical performance gaps: GPT-4.1 1 achieves only 52.11% accuracy, indicating substantial room for improvement.The dataset and evaluation code 2 are publicly available.

Authors

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.findings-acl.1083
Citations
1
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
2.48

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

C²RBench: A Chinese Complex Reasoning Benchmark for Large Language Models

Junru Wu, Deyi Xiong, Linxi Su, Tianhao Shen
1 citations
Topic Modeling
2.48
article

C²RBench: A Chinese Complex Reasoning Benchmark for Large Language Models

Junru Wu, Deyi Xiong, Linxi Su, Tianhao Shen
article en
1 citations

Abstract

Large language models (LLMs) have achieved remarkable progress in autonomous reasoning, evolving from basic text processing to sophisticated multimodal reasoning, a critical capability for general-purpose AI assistants.However, existing benchmarks usually fail to adequately capture the intricate multi-step reasoning demands inherent in real-world scenarios.To bridge this gap, we propose C 2 RBench: a Chinese Complex Reasoning Benchmark for evaluating multi-step, multimodal advanced reasoning capability of LLMs.C 2 RBench comprises 1,115 carefully curated Chinese tasks, which are organized into eight domain-specific subsets, each meticulously designed to mirror real-world challenges.This hierarchical benchmark features three difficulty tiers based on the number of reasoning steps required (average 8.44 steps per task), significantly exceeding existing benchmarks in cognitive complexity.Extensive evaluations of 20 LLMs (including DeepSeek-R1) and 24 multimodal large language models (MLLMs) on C 2 RBench reveal critical performance gaps: GPT-4.1 1 achieves only 52.11% accuracy, indicating substantial room for improvement.The dataset and evaluation code 2 are publicly available.

National Key Research and Development Program of China, Moonshot Research and Development Program
Quality Education
Openalex Percentile: Top 9%
Topic Modeling
2.48
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.