Over-Reasoning in Table Question Answering: A Behavioral Analysis of Chain-of-Thought Models
Reasoning models trained with extended chain-of-thought (CoT) objectives excel on mathematical benchmarks, yet their suitability for structured data tasks remains underexplored. This paper presents a systematic behavioral study of reasoning versus instruction-tuned models on table question answering (TableQA) across three benchmarks and four question types. We identify a phenomenon we term over-reasoning: the reasoning model (Qwen3-32B) generates traces 5–13× longer than baselines yet achieves lower overall accuracy, with accuracy declining monotonically from 91.0% (short responses) to 64.8% (long responses). We introduce two trace-level analyses. First, a faithfulness analysis shows that when the reasoning trace does not contain the ground-truth answer, accuracy collapses to 9.2%, while faithful traces reach 95.2%. Second, a trace surgery experiment reveals that over half of each trace is generated after the answer has already been located. Finally, structured step-by-step prompting improves instruction-tuned baselines by up to 11.8 percentage points but hurts the reasoning model (−1.7 pp), suggesting that the failure stems from a mismatch between sequential derivation objectives and the discrete structural operations required by tabular reasoning.
Authors
- MAITREYA PRASAD WARRIER
- NEHAA SHRI MS (ORCID: https://orcid.org/0009-0001-9304-3039)
Institutions
- Vellore Institute of Technology University (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.5281/zenodo.24283874
- Primary Topic
- Topic Modeling
- Type
- preprint