Policy Iteration Convergence for Exploratory Two-State Mean-Field Control Problems

This paper studies the theoretical policy iteration convergence for two-state mean-field control (MFC) problems with Wright--Fisher common noise under entropy regularization. The population dynamics lead to a degenerate Hamilton--Jacobi--Bellman equation of Kimura type. After an arcsine transformation, the degeneracy becomes a singular Bessel drift whose coefficients and weights depend on the feedback stochastic policy. This policy dependence prevents a direct application of estimates for a fixed operator. We establish weighted Sobolev estimates and Hölder embeddings uniformly over the admissible policy class. Combining these estimates with logarithmic entropy bounds and boundary estimates that remain valid when the policy changes, we prove the well-posedness, monotone improvement and convergence of policy iteration to the optimal solution. We also establish the convergence to the original MFC problem when the entropy parameter $λ$ tends to zero. Both the regularization error and the performance loss of the regularized optimal policy are bounded by $O(λ(1+|\lnλ|))$ in a weighted Hölder norm. An epidemic containment example of MFC is presented to illustrate the policy iteration convergence and the vanishing entropy convergence.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-16
DOI
https://doi.org/10.48550/arxiv.2611.21122
Primary Topic
Stochastic processes and financial applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Policy Iteration Convergence for Exploratory Two-State Mean-Field Control Problems

Xiaoli Wei, Zhenhua Wang, Xiang Yu
arXiv (Cornell University)
Stochastic processes and financial applications
preprint

Policy Iteration Convergence for Exploratory Two-State Mean-Field Control Problems

Xiaoli Wei, Zhenhua Wang, Xiang Yu
preprint en

Abstract

This paper studies the theoretical policy iteration convergence for two-state mean-field control (MFC) problems with Wright--Fisher common noise under entropy regularization. The population dynamics lead to a degenerate Hamilton--Jacobi--Bellman equation of Kimura type. After an arcsine transformation, the degeneracy becomes a singular Bessel drift whose coefficients and weights depend on the feedback stochastic policy. This policy dependence prevents a direct application of estimates for a fixed operator. We establish weighted Sobolev estimates and Hölder embeddings uniformly over the admissible policy class. Combining these estimates with logarithmic entropy bounds and boundary estimates that remain valid when the policy changes, we prove the well-posedness, monotone improvement and convergence of policy iteration to the optimal solution. We also establish the convergence to the original MFC problem when the entropy parameter $λ$ tends to zero. Both the regularization error and the performance loss of the regularized optimal policy are bounded by $O(λ(1+|\lnλ|))$ in a weighted Hölder norm. An epidemic containment example of MFC is presented to illustrate the policy iteration convergence and the vanishing entropy convergence.

arXiv (Cornell University)
Stochastic processes and financial applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon