Naming the deliverable raises convention coverage in large language model outputs

Abstract Practitioners compress conventions into a name such as RACI matrix or term sheet, whereas outsiders can only describe the deliverable. We tested whether the name matters when large language models write professional deliverables. Each request had three versions differing in one phrase, the practitioner name or one of two lay descriptions. Blind model judges scored outputs against convention checklists from public normative sources. On 40 curated terms, the name raised convention coverage in all six models, by 0.11 [95% CI 0.07, 0.16] to 0.29 [0.22, 0.37], and the gain survived masking the name, rebuilt checklists, and length control. On 150 terms sampled by rule from O*NET and Wikidata, it replicated in four models (0.11 [0.08, 0.14] to 0.19 [0.16, 0.23]) and was largest for method names. Gains were largest in models with the lowest lay coverage, a pattern confounded with vendor and output length. On curated terms, log-probabilities from an open model correlated moderately with per-term gains (ρ = 0.37). Using a name obtained from the model recovered about half to three quarters of the gain, mostly when the name was correct. We propose knowing that deliverables have names, asking for them, and checking them as an AI-literacy skill.

Authors

Publication Details

Journal
Research Square
Published
2026-11-04
DOI
https://doi.org/10.21203/rs.3.rs-11602543/v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Naming the deliverable raises convention coverage in large language model outputs

Jiawei Jing
Research Square
Topic Modeling
preprint

Naming the deliverable raises convention coverage in large language model outputs

Jiawei Jing
preprint en

Abstract

Abstract Practitioners compress conventions into a name such as RACI matrix or term sheet, whereas outsiders can only describe the deliverable. We tested whether the name matters when large language models write professional deliverables. Each request had three versions differing in one phrase, the practitioner name or one of two lay descriptions. Blind model judges scored outputs against convention checklists from public normative sources. On 40 curated terms, the name raised convention coverage in all six models, by 0.11 [95% CI 0.07, 0.16] to 0.29 [0.22, 0.37], and the gain survived masking the name, rebuilt checklists, and length control. On 150 terms sampled by rule from O*NET and Wikidata, it replicated in four models (0.11 [0.08, 0.14] to 0.19 [0.16, 0.23]) and was largest for method names. Gains were largest in models with the lowest lay coverage, a pattern confounded with vendor and output length. On curated terms, log-probabilities from an open model correlated moderately with per-term gains (ρ = 0.37). Using a name obtained from the model recovered about half to three quarters of the gain, mostly when the name was correct. We propose knowing that deliverables have names, asking for them, and checking them as an AI-literacy skill.

Research Square
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon