Preregistration: Study 3 of 'Searching the hidden inventory' - which rules of a frozen annotation harness carry its gain (TalkMoves Subset 2)

Test of what carries the gain of a frozen annotation harness for LLM coding of teacher talk moves. Nothing is optimized. Claude Sonnet 5.5 codes 104 new passages from 15 teachers of TalkMoves Subset 2 with twelve instruments: the harness, the manual, the manual plus one class of rules, and the harness without one class of rules. Classes are rules already in the manual, operationalizations, added conventions and rules in tension with the manual. Confirmatory hypotheses compare added conventions with repetitions of the manual in both directions, with a teacher-level cluster bootstrap and Holm correction. Registered before the passages were drawn. The deposit includes all instrument texts except the manual.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-02
DOI
https://doi.org/10.5281/zenodo.23891997
Primary Topic
Computational and Text Analysis Methods
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Preregistration: Study 3 of 'Searching the hidden inventory' - which rules of a frozen annotation harness carry its gain (TalkMoves Subset 2)

Simon Filler
5 links
Zenodo (CERN European Organization for Nuclear Research)
Computational and Text Analysis Methods
article

Preregistration: Study 3 of 'Searching the hidden inventory' - which rules of a frozen annotation harness carry its gain (TalkMoves Subset 2)

Simon Filler
article en

Abstract

Test of what carries the gain of a frozen annotation harness for LLM coding of teacher talk moves. Nothing is optimized. Claude Sonnet 5.5 codes 104 new passages from 15 teachers of TalkMoves Subset 2 with twelve instruments: the harness, the manual, the manual plus one class of rules, and the harness without one class of rules. Classes are rules already in the manual, operationalizations, added conventions and rules in tension with the manual. Confirmatory hypotheses compare added conventions with repetitions of the manual in both directions, with a teacher-level cluster bootstrap and Holm correction. Registered before the passages were drawn. The deposit includes all instrument texts except the manual.

Zenodo (CERN European Organization for Nuclear Research)
TU Dortmund University (DE)
Openalex Percentile: Top 7%
Computational and Text Analysis Methods
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon