Read Routing Entropy at a Fixed Position: A Cross-Model Study of the Prefix Effect in Mixture-of-Experts Routing

Routing entropy is increasingly used as a signal for adaptive compute, safety telemetry, and expert specialization. Across recent uses of routing entropy that we surveyed, routing statistics are typically pooled across token positions or reduced to sequence-level measures without an explicit control for token position or prompt length. The magnitude of the positional contribution to these measurements therefore remains unknown. We measured per-token routing entropy in four mixture-of-experts runs, Qwen3.5-35B-A3B, Qwen3.5-397B-A17B, DeepSeek-V3-0324 and DeepSeek R1 and compared position-averaged readouts against fixed-position readouts on the same prompts. Full position curves were retained for five prompts on one model and for 301 raw-text windows on another. The positional term in the router is a prefix effect. Per-token routing entropy is depressed over the chat template and roughly the opening tens of positions and is flat after that. On raw text with no template the depression is confined to the first token. An all-token average therefore dilutes a fixed low-entropy prefix in proportion to prompt length; on the same captures, the all-token average orders a graded prompt suite by length, and the fixed-position readout removes that order and, at the extremes, reverses it. We recommend treating token position as a default covariate in any routing statistic, reporting the readout position alongside the value, and preferring a fixed-position or position-matched readout over an all-token average when prompts of different lengths are compared. Under that control, a difference in routing entropy is a difference in routing. Version 2.0 (2026-08-28) supersedes version 1.1 ("Sequence Position Explains an Apparent Complexity Gradient in Mixture-of-Experts Routing Entropy"). It is a new paper on the same runs: the opening-depression finding is stated as a prefix effect and tested on a full-precision raw-text run (Run 4, Qwen3.5-35B-A3B-Base, 301 windows); the two-readout disagreement is reported with bootstrap intervals; the chat-template markers are documented as literal text; and every reported value regenerates from the public data bundle. Correction to version 1.1: the checkpoint reported there as DeepSeek V3.1 was DeepSeek-V3-0324 (Unsloth UD-Q2_K_XL); the capture command and archive record load DeepSeek-V3-0324. Data and code: https://huggingface.co/datasets/ec75hash/routing-entropy-token-position