Reinjection Does Not Decide Presence: Occupancy and Identifiability under Mixed Multinomial Retraining
A categorical model retrained at temperature one replaces a sample of size \(n\) by \(n_r\) fresh draws from a fixed seed and \(n_s=n-n_r\) draws from the previous frequencies. For one seed symbol of mass \(p\), the count is the Markov chain \(X'=\operatorname{Binomial}(n_s,X/n)+\operatorname{Binomial}(n_r,p)\). The symbol is absent with probability at most \((1-p)^{n_r}\), at every generation and in stationarity, and its stationary mean count is exactly \(np\). Occupancy is therefore bounded from below by the fresh copies and from above by the whole sample. The expression \(\alpha=1/(np)\) has the units of a critical mix, and \(n_rp\ge 1\) is a sufficient scale for presence on the grid we ran, but occupancy is not a function of \(n_rp\) alone: at reinjection \(0.105\), measured presence is \(0.24\) when \(np=0.42\) and \(0.70\) when \(np \approx 42\). At a known \((n, n_r)\), the occupancy map is strictly increasing in \(p\), with difference quotients at most \(n\), and it can be inverted where the slope stays away from zero. On a Zipf alphabet the inverse recovers \(p\) to a median factor of about \(1.07\) on that band, and it returns nothing once every symbol is saturated. In the rare-count regime the stationary occupancy converges to that of a subcritical branching process with immigration. The forced-presence bound uses only that the fresh draws are independent of the synthetic tokens and that a probability is at most one, so dependence among the synthetic tokens, including an autoregressive continuation, does not remove it. The state of the recursion is the frequency vector. Its categorical Fisher matrix \(\operatorname{diag}(p)-pp^{\top}\) has participation ratio zero at a vertex, while a Gaussian location term scales as \(n/\sigma^{2}\) and grows when a surviving component sharpens. Any scalar that adds those two blocks moves in opposite directions under mode death and under sharpening.
Authors
- Sathwik Garikapati (ORCID: https://orcid.org/0009-0000-3159-7485)
Publication Details
- Journal
- Preprints.org
- Published
- 2026-10-27
- DOI
- https://doi.org/10.20944/preprints202610.2604.v1
- Primary Topic
- Probability and Statistical Research
- Type
- preprint