Designing asymptotically optimal policies for continuous-time weakly coupled MDPs

We study the continuous-time Weakly Coupled Markov Decision Process (WCMDP), a class of decision problems involving multiple interacting Markov processes (or ''arms'') subject to shared resource constraints. We present a general framework for policy design using a combination of an underlying Markov process and a sequence of mappings. Our main theoretical result establishes suffifient conditions for asymptotic optimality as the number of arms grows. We construct both deterministic and randomized policies based on a solution to a linear program (LP). These policies initially assign actions to arms—either proportionally (deterministic) or randomly—based on conditional measures derived from the LP. As this initial allocation may violate feasibility constraints, we introduce a mapping to enforce the resource constraints are satisfied.