Automatic Functional Characterization of Chains of Thought Generated by Reasoning Models

Reasoning models generate chains of thought (gCoT) that externalize deliberations, calculations, verifications, corrections and alternatives, and are today a principal source of information about their behavior. The literature examines their faithfulness to internal computation, answer correctness, length and step quality, or describes their structure with step-level graphs and trees; multi-step operations and their relations remain less characterized. This work proposes describing that organization through a functional profile obtained with an automatic, language-model-supported method that represents each chain as a graph, identifies its episodes and summarizes their relations. Validity and reliability were tested: on a control corpus the method detected all explicit deliberative episodes and 71% of implicit ones; on real chains, group means of 25 chains proved reliable. In 200 chains from four models on shared medical and legal problems, organization differed between models and was similar across domains; verifications rarely refuted (8%) and alternatives were rarely adopted (2%). The results suggest that the profile captures each model’s features: it allowed the generating model to be identified in 67% of chains, versus 49% with correctness, length and domain, and added no detectable information about correctness. Profiles describe the observable chain organization at the group level, not the models’ internal process.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-12
DOI
https://doi.org/10.20944/preprints202611.1343.v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Automatic Functional Characterization of Chains of Thought Generated by Reasoning Models

John W Branch-Bedoya, Luis F. Becerra-Monsalve, German Sanchez-Torres
Preprints.org
Topic Modeling
preprint

Automatic Functional Characterization of Chains of Thought Generated by Reasoning Models

John W Branch-Bedoya, Luis F. Becerra-Monsalve, German Sanchez-Torres
preprint en

Abstract

Reasoning models generate chains of thought (gCoT) that externalize deliberations, calculations, verifications, corrections and alternatives, and are today a principal source of information about their behavior. The literature examines their faithfulness to internal computation, answer correctness, length and step quality, or describes their structure with step-level graphs and trees; multi-step operations and their relations remain less characterized. This work proposes describing that organization through a functional profile obtained with an automatic, language-model-supported method that represents each chain as a graph, identifies its episodes and summarizes their relations. Validity and reliability were tested: on a control corpus the method detected all explicit deliberative episodes and 71% of implicit ones; on real chains, group means of 25 chains proved reliable. In 200 chains from four models on shared medical and legal problems, organization differed between models and was similar across domains; verifications rarely refuted (8%) and alternatives were rarely adopted (2%). The results suggest that the profile captures each model’s features: it allowed the generating model to be identified in 67% of chains, versus 49% with correctness, length and domain, and added no detectable information about correctness. Profiles describe the observable chain organization at the group level, not the models’ internal process.

Preprints.org
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon