The Compatible Law of Full Histories: Three Quantifiers and the Complete Class of Optimal Policies
Can a single policy optimize the entire distribution of the smallest margin attained along an infinite history? We study this question on compact metric state spaces with closed succession relations, nonempty successor sets, and nonnegative upper semicontinuous margins. Three distinct requirements organize the analysis: the existence of a satisfactory history, almost-sure satisfaction under a given execution law, and a guarantee along every admissible branch. When all measurable choices allowed by the succession relations are available, a Bellman selector generates a single path law that simultaneously maximizes every threshold survival probability, every quantile, and the expectation of every nonnegative nondecreasing function of the margin. We characterize the complete class of Markov kernels attaining this profile from every starting time and state: their transitions must be concentrated on successors preserving the Bellman value. For restricted families of available kernels, this condition determines attainability of the geometric optimum, while optimal policies may otherwise depend on the chosen threshold. An exact loss identity equates the integrated difference between the optimal and realized threshold profiles with both the expected margin loss and the first Wasserstein distance between their distributions. Local Bellman defects and departures from optimal successor sets yield sharp quantitative loss bounds. An essential-infimum recursion locates almost-sure performance between existential and universal guarantees. Finally, coupling the common parts of transition kernels gives total-variation bounds for complete path laws, threshold-profile estimates, and quantile stability under stated regularity conditions. An explicit example identifies the discontinuity of almost-sure performance under an arbitrarily small probability of entering a bad branch.
Authors
- Maciej Stachowiak
- Paweł Nowak (ORCID: https://orcid.org/0009-0000-5562-1128)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24107394
- Primary Topic
- Adaptive Dynamic Programming Control
- Type
- preprint