Naming the deliverable raises convention coverage in large language model outputs
Abstract Practitioners compress conventions into a name such as RACI matrix or term sheet, whereas outsiders can only describe the deliverable. We tested whether the name matters when large language models write professional deliverables. Each request had three versions differing in one phrase, the practitioner name or one of two lay descriptions. Blind model judges scored outputs against convention checklists from public normative sources. On 40 curated terms, the name raised convention coverage in all six models, by 0.11 [95% CI 0.07, 0.16] to 0.29 [0.22, 0.37], and the gain survived masking the name, rebuilt checklists, and length control. On 150 terms sampled by rule from O*NET and Wikidata, it replicated in four models (0.11 [0.08, 0.14] to 0.19 [0.16, 0.23]) and was largest for method names. Gains were largest in models with the lowest lay coverage, a pattern confounded with vendor and output length. On curated terms, log-probabilities from an open model correlated moderately with per-term gains (ρ = 0.37). Using a name obtained from the model recovered about half to three quarters of the gain, mostly when the name was correct. We propose knowing that deliverables have names, asking for them, and checking them as an AI-literacy skill.
Authors
- Jiawei Jing (ORCID: https://orcid.org/0009-0000-8365-367X)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-04
- DOI
- https://doi.org/10.21203/rs.3.rs-11602543/v1
- Primary Topic
- Topic Modeling
- Type
- preprint