Preregistration: Study 2 of 'An LLM manual for coding teacher talk moves' - estimation and replication of a frozen annotation harness on unseen teachers (TalkMoves Subset 2)
Estimation and replication study: the frozen annotation harness from Study 1 is applied without further optimization to 125 passages from 19 teachers of TalkMoves Subset 2 (TalkBack) that were not used in Study 1. Primary estimand: Cohen's kappa against human gold with a teacher-level cluster bootstrap CI for Claude Fable 5.1 coding sentence by sentence; secondary: harness, model, unit and model-family effects (Claude Sonnet 5.5, GPT-6 Luna) and replication of Study 1. Registered before the passages were drawn.
Authors
- Simon Filler (ORCID: https://orcid.org/0009-0008-8736-8831)
Institutions
- TU Dortmund University (DE)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-02
- DOI
- https://doi.org/10.5281/zenodo.23880253
- Primary Topic
- Computational and Text Analysis Methods
- Type
- article
- Field-Weighted Citation Impact
- 0.00