Authorship Drift: A Multi-Turn Evaluation of Direction Substitution in AI Collaboration. A Study Protocol

This protocol pre-registers an evaluation of authorship drift, defined as multi-turn direction substitution in human-AI collaboration. It measures direction substitution in long, fluent collaboration: the point where a suggestion the model made becomes something the user believes they thought. Introspection is unreliable here, which is why it has to be instrumented externally, at the transcript level, rather than asked about in a survey. Within this protocol the construct is accordingly operationalized at the transcript level, as unmarked change of the user's stated direction scored on model behavior; the user-belief question belongs to a separately registered companion study. Changed thinking is not the harm; the lost choice point is: a labeled contribution puts a decision in front of the user, and an unlabeled one is absorbed without a decision. Each run of each scenario is scored on five pass/fail dimensions: whether the model's contribution changed the user's stated direction, whether that contribution was presented as the model's own or as continuous with the user's material, whether the model surfaced the change against the user's earlier stated intent, whether the model answers accurately when the user asks "was that mine or yours," and whether it restores the user's direction cleanly when they push back. The suite also scores the opposite failure: a model that withholds or over-hedges genuine creative help. The candidate fix is labeled contribution, not less contribution. Alongside the pass/fail dimensions, each finished artifact gets a provenance ratio: the share of the finished artifact originating in the model's contributions rather than in the user's seed (measured at the requirement level, Section 4.5). Comparing that ratio between labeled and unlabeled runs grounds the test of whether labeling reduces absorption, rather than assuming it. Within this protocol that comparison is descriptive; the causal test in people belongs to the planned companion study. Each scenario is run five times per model; dimension results are reported as pass rates per scenario, and run-to-run inconsistency is itself reported as a finding. Scenarios are multi-turn conversations in which a simulated user, disclosed as simulated, states a direction early and the model under test responds live. Grading uses a model from a family not under test, scoring each dimension independently in single-output pass/fail calls (no pairwise comparison, so pairwise position bias does not arise), calibrated against human labels, with agreement reported. The protocol, rubric, grader prompts, scoring code, human labels, results, and the scenario-construction method will be released openly; generated scenario items are embargoed at registration and regenerable from the published method; scenario seeds remain private. This registration covers a pilot of 25 scenarios across three domains (reflective and memoir writing, personal decisions, and AI-assisted design and build work), run against three model families; the core suite (100+ scenarios, four model families) and a companion study with human participants are planned extensions, each to be registered separately. Study protocol, version of record 1.0 (2026-11-11). OSF preregistration to follow, citing this deposit. Text and data CC BY 4.0; code MIT.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-11
DOI
https://doi.org/10.5281/zenodo.24147412
Primary Topic
Usability and User Interface Design
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Authorship Drift: A Multi-Turn Evaluation of Direction Substitution in AI Collaboration. A Study Protocol

Marissa Mellor Gates
Zenodo (CERN European Organization for Nuclear Research)
Usability and User Interface Design
preprint

Authorship Drift: A Multi-Turn Evaluation of Direction Substitution in AI Collaboration. A Study Protocol

Marissa Mellor Gates
preprint en

Abstract

This protocol pre-registers an evaluation of authorship drift, defined as multi-turn direction substitution in human-AI collaboration. It measures direction substitution in long, fluent collaboration: the point where a suggestion the model made becomes something the user believes they thought. Introspection is unreliable here, which is why it has to be instrumented externally, at the transcript level, rather than asked about in a survey. Within this protocol the construct is accordingly operationalized at the transcript level, as unmarked change of the user's stated direction scored on model behavior; the user-belief question belongs to a separately registered companion study. Changed thinking is not the harm; the lost choice point is: a labeled contribution puts a decision in front of the user, and an unlabeled one is absorbed without a decision. Each run of each scenario is scored on five pass/fail dimensions: whether the model's contribution changed the user's stated direction, whether that contribution was presented as the model's own or as continuous with the user's material, whether the model surfaced the change against the user's earlier stated intent, whether the model answers accurately when the user asks "was that mine or yours," and whether it restores the user's direction cleanly when they push back. The suite also scores the opposite failure: a model that withholds or over-hedges genuine creative help. The candidate fix is labeled contribution, not less contribution. Alongside the pass/fail dimensions, each finished artifact gets a provenance ratio: the share of the finished artifact originating in the model's contributions rather than in the user's seed (measured at the requirement level, Section 4.5). Comparing that ratio between labeled and unlabeled runs grounds the test of whether labeling reduces absorption, rather than assuming it. Within this protocol that comparison is descriptive; the causal test in people belongs to the planned companion study. Each scenario is run five times per model; dimension results are reported as pass rates per scenario, and run-to-run inconsistency is itself reported as a finding. Scenarios are multi-turn conversations in which a simulated user, disclosed as simulated, states a direction early and the model under test responds live. Grading uses a model from a family not under test, scoring each dimension independently in single-output pass/fail calls (no pairwise comparison, so pairwise position bias does not arise), calibrated against human labels, with agreement reported. The protocol, rubric, grader prompts, scoring code, human labels, results, and the scenario-construction method will be released openly; generated scenario items are embargoed at registration and regenerable from the published method; scenario seeds remain private. This registration covers a pilot of 25 scenarios across three domains (reflective and memoir writing, personal decisions, and AI-assisted design and build work), run against three model families; the core suite (100+ scenarios, four model families) and a companion study with human participants are planned extensions, each to be registered separately. Study protocol, version of record 1.0 (2026-11-11). OSF preregistration to follow, citing this deposit. Text and data CC BY 4.0; code MIT.

Zenodo (CERN European Organization for Nuclear Research)
Usability and User Interface Design
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon