Over-Reasoning in Table Question Answering: A Behavioral Analysis of Chain-of-Thought Models

Reasoning models trained with extended chain-of-thought (CoT) objectives excel on mathematical benchmarks, yet their suitability for structured data tasks remains underexplored. This paper presents a systematic behavioral study of reasoning versus instruction-tuned models on table question answering (TableQA) across three benchmarks and four question types. We identify a phenomenon we term over-reasoning: the reasoning model (Qwen3-32B) generates traces 5–13× longer than baselines yet achieves lower overall accuracy, with accuracy declining monotonically from 91.0% (short responses) to 64.8% (long responses). We introduce two trace-level analyses. First, a faithfulness analysis shows that when the reasoning trace does not contain the ground-truth answer, accuracy collapses to 9.2%, while faithful traces reach 95.2%. Second, a trace surgery experiment reveals that over half of each trace is generated after the answer has already been located. Finally, structured step-by-step prompting improves instruction-tuned baselines by up to 11.8 percentage points but hurts the reasoning model (−1.7 pp), suggesting that the failure stems from a mismatch between sequential derivation objectives and the discrete structural operations required by tabular reasoning.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24283874
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Over-Reasoning in Table Question Answering: A Behavioral Analysis of Chain-of-Thought Models

MAITREYA PRASAD WARRIER, NEHAA SHRI MS
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Over-Reasoning in Table Question Answering: A Behavioral Analysis of Chain-of-Thought Models

MAITREYA PRASAD WARRIER, NEHAA SHRI MS
preprint en

Abstract

Reasoning models trained with extended chain-of-thought (CoT) objectives excel on mathematical benchmarks, yet their suitability for structured data tasks remains underexplored. This paper presents a systematic behavioral study of reasoning versus instruction-tuned models on table question answering (TableQA) across three benchmarks and four question types. We identify a phenomenon we term over-reasoning: the reasoning model (Qwen3-32B) generates traces 5–13× longer than baselines yet achieves lower overall accuracy, with accuracy declining monotonically from 91.0% (short responses) to 64.8% (long responses). We introduce two trace-level analyses. First, a faithfulness analysis shows that when the reasoning trace does not contain the ground-truth answer, accuracy collapses to 9.2%, while faithful traces reach 95.2%. Second, a trace surgery experiment reveals that over half of each trace is generated after the answer has already been located. Finally, structured step-by-step prompting improves instruction-tuned baselines by up to 11.8 percentage points but hurts the reasoning model (−1.7 pp), suggesting that the failure stems from a mismatch between sequential derivation objectives and the discrete structural operations required by tabular reasoning.

Zenodo (CERN European Organization for Nuclear Research)
Vellore Institute of Technology University (IN)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon