Beyond Second Chances: A Transition Analysis of Peer Feedback in LLM Revision

When multiple LLM-generated reasoning paths independently answer a question and then revise after exchanging peer responses, aggregate accuracy sometimes improves and sometimes declines. We introduce a paired-transition framework that decomposes revision-stage gains into a retry baseline and two peer-feedback channels: an exposure channel (behavioral shift from generic, task-irrelevant peer context) and an evidence channel (incremental repair from task-relevant rationales). A paired ablation with up to six conditions across three tasks (Logic Grid Puzzle, GSM8K, FOLIO) and three open-weight model families (Gemma-4-31B, Qwen3.6-27B, Mistral-Small-3.2-24B), with pre-registered hypotheses and Holm-corrected bootstrap tests, reveals that the evidence channel provides a robust positive repair signal across all confirmatory families, while the exposure channel is model-dependent---ranging from mildly stabilizing to strongly destabilizing---falsifying the hypothesis that peer exposure universally reduces revision damage. A cross-size peer-source swap on Logic Grid shows that peer-trace quality can be the dominant bottleneck in evidence-rich settings. An adjudicated blinded-coding study provides convergent evidence that successful repairs more often contain novel reasoning steps than matched controls. These findings transform the question of whether peer revision helps into a structural analysis of which feedback components work, when, and why.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-30
DOI
https://doi.org/10.5281/zenodo.21045135
Primary Topic
Decision-Making and Behavioral Economics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Beyond Second Chances: A Transition Analysis of Peer Feedback in LLM Revision

Chien-Tung Lin
Zenodo (CERN European Organization for Nuclear Research)
Decision-Making and Behavioral Economics
preprint

Beyond Second Chances: A Transition Analysis of Peer Feedback in LLM Revision

Chien-Tung Lin
preprint en

Abstract

When multiple LLM-generated reasoning paths independently answer a question and then revise after exchanging peer responses, aggregate accuracy sometimes improves and sometimes declines. We introduce a paired-transition framework that decomposes revision-stage gains into a retry baseline and two peer-feedback channels: an exposure channel (behavioral shift from generic, task-irrelevant peer context) and an evidence channel (incremental repair from task-relevant rationales). A paired ablation with up to six conditions across three tasks (Logic Grid Puzzle, GSM8K, FOLIO) and three open-weight model families (Gemma-4-31B, Qwen3.6-27B, Mistral-Small-3.2-24B), with pre-registered hypotheses and Holm-corrected bootstrap tests, reveals that the evidence channel provides a robust positive repair signal across all confirmatory families, while the exposure channel is model-dependent---ranging from mildly stabilizing to strongly destabilizing---falsifying the hypothesis that peer exposure universally reduces revision damage. A cross-size peer-source swap on Logic Grid shows that peer-trace quality can be the dominant bottleneck in evidence-rich settings. An adjudicated blinded-coding study provides convergent evidence that successful repairs more often contain novel reasoning steps than matched controls. These findings transform the question of whether peer revision helps into a structural analysis of which feedback components work, when, and why.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Decision-Making and Behavioral Economics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.