Behavioral Geometry of Large Language Models

Large language models return different outputs when the same question is posed repeatedly. This study presents a framework that treats LLMs as a distribution sampling device and the responses it generates as drawn from that distribution. On a set question bank each model:question pair was summarized across ten stochastic trials. The summary was used to assign an integer coordinate of correct answers, abstentions, and distinct wrong answers locating the pair within a bounded three-dimensional outcome space. Prompt variations displace the coordinates assigned and that displacement was measured as a geometric vector against a baseline noise floor. A bank of 1,432 questions was assembled from public benchmark question sets after form normalization across seventeen topics. Six hosted models from three vendor families and one local open-weight model were evaluated under closed-book conditions. The hosted set included three in-family scaling pairs. Three single-clause prompt variations and one composite prompt were applied. Baseline run-to-run displacement was practically equivalent to zero, which established a measurable noise floor. Format constraints reduced output length but lowered the user-defined value, negatively impacting abstention, and reordered the accuracy ranking. An abstention license alone raised or held that value for every model. The composite prompt resolved differently across models, moving some models toward better value and others toward worse. Effect magnitude tracked each model's baseline abstention surface rather than the instruction content. A prompt variation informed by the framework that re-ordered the two clauses improved performance for all six hosted models. These results indicate that prompt intervention is not transferable across models. The framework separates a descriptive displacement layer from a user-defined value layer and the separation supports per-model behavioral characterization depth that a single scalar score cannot provide

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-16
DOI
https://doi.org/10.5281/zenodo.20723177
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Behavioral Geometry of Large Language Models

Kyle Botsch
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Behavioral Geometry of Large Language Models

Kyle Botsch
preprint en

Abstract

Large language models return different outputs when the same question is posed repeatedly. This study presents a framework that treats LLMs as a distribution sampling device and the responses it generates as drawn from that distribution. On a set question bank each model:question pair was summarized across ten stochastic trials. The summary was used to assign an integer coordinate of correct answers, abstentions, and distinct wrong answers locating the pair within a bounded three-dimensional outcome space. Prompt variations displace the coordinates assigned and that displacement was measured as a geometric vector against a baseline noise floor. A bank of 1,432 questions was assembled from public benchmark question sets after form normalization across seventeen topics. Six hosted models from three vendor families and one local open-weight model were evaluated under closed-book conditions. The hosted set included three in-family scaling pairs. Three single-clause prompt variations and one composite prompt were applied. Baseline run-to-run displacement was practically equivalent to zero, which established a measurable noise floor. Format constraints reduced output length but lowered the user-defined value, negatively impacting abstention, and reordered the accuracy ranking. An abstention license alone raised or held that value for every model. The composite prompt resolved differently across models, moving some models toward better value and others toward worse. Effect magnitude tracked each model's baseline abstention surface rather than the instruction content. A prompt variation informed by the framework that re-ordered the two clauses improved performance for all six hosted models. These results indicate that prompt intervention is not transferable across models. The framework separates a descriptive displacement layer from a user-defined value layer and the separation supports per-model behavioral characterization depth that a single scalar score cannot provide

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.