Certified Nash Equilibrium Learning for Nonzero-Sum Jump-Diffusion Games: Policy Iteration and Off-Policy Q-Learning

This paper establishes two model-free reinforcement learning methods for infinite-horizon N-player linear-quadratic nonzero-sum stochastic differential games whose drift, Brownian diffusion, and Poisson jump amplitudes depend on all players' controls. Control dependence in the diffusion and jump terms couples the players' simultaneous feedback updates. A verifiable certificate ensures that the partially model-free policy iteration map is contractive on an invariant closed ball centered at an admissible initial feedback profile. All feedback iterates are mean-square stabilizing, and the algorithm achieves local geometric convergence of feedback gains and value matrices to those associated with a closed-loop Nash equilibrium. The term certified refers to a verifiable convergence certificate that guarantees mean-square stability of all policy iterates and local geometric convergence to a closed-loop Nash equilibrium. To implement the policy updates from data, the off-policy Q-learning algorithm reuses the same behavior-policy data across iterations without requiring the system coefficients or jump intensity. Under quantitative persistence of excitation and a common initialization, exact population regression reproduces the policy-iteration sequence. The same local convergence guarantee therefore applies to off-policy Q-learning at the population level. Simulation results demonstrate the effectiveness of the two proposed algorithms.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-16
DOI
https://doi.org/10.48550/arxiv.2611.21272
Primary Topic
Adaptive Dynamic Programming Control
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Certified Nash Equilibrium Learning for Nonzero-Sum Jump-Diffusion Games: Policy Iteration and Off-Policy Q-Learning

Tielong Shen, Zuo Quan Xu, Na Li, Pengwei Wang et al.
arXiv (Cornell University)
Adaptive Dynamic Programming Control
preprint

Certified Nash Equilibrium Learning for Nonzero-Sum Jump-Diffusion Games: Policy Iteration and Off-Policy Q-Learning

Tielong Shen, Zuo Quan Xu, Na Li, Pengwei Wang, Jiacheng Fan
preprint en

Abstract

This paper establishes two model-free reinforcement learning methods for infinite-horizon N-player linear-quadratic nonzero-sum stochastic differential games whose drift, Brownian diffusion, and Poisson jump amplitudes depend on all players' controls. Control dependence in the diffusion and jump terms couples the players' simultaneous feedback updates. A verifiable certificate ensures that the partially model-free policy iteration map is contractive on an invariant closed ball centered at an admissible initial feedback profile. All feedback iterates are mean-square stabilizing, and the algorithm achieves local geometric convergence of feedback gains and value matrices to those associated with a closed-loop Nash equilibrium. The term certified refers to a verifiable convergence certificate that guarantees mean-square stability of all policy iterates and local geometric convergence to a closed-loop Nash equilibrium. To implement the policy updates from data, the off-policy Q-learning algorithm reuses the same behavior-policy data across iterations without requiring the system coefficients or jump intensity. Under quantitative persistence of excitation and a common initialization, exact population regression reproduces the policy-iteration sequence. The same local convergence guarantee therefore applies to off-policy Q-learning at the population level. Simulation results demonstrate the effectiveness of the two proposed algorithms.

arXiv (Cornell University)
Adaptive Dynamic Programming Control
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon