Advantage Alignment Does Not Preserve Mixed Nash Equilibria: A Machine-Checked Counterexample and an Exact Covariance Characterization
Advantage Alignment (Duque et al., ICLR 2025) states that its alignment term vanishes at every Nash equilibrium. We construct a two-stage game whose unique Markov-perfect equilibrium is completely mixed and at which the update is nonzero; after one step the other player has a strictly profitable deviation. The alignment term equals a pre-action weight times the covariance between the agent's score and the welfare externality of its action on the opponent; the update is not invariant to strategic equivalence; and in normal-form games AA reads only the opponent's welfare externality whereas first-order LOLA reads only the interaction that changes the opponent's incentives. Main statements are machine-checked in Lean 4 with Mathlib. Advantage Alignment, its Equation 8 and Theorem 3 are the work of Duque, Aghajohari, Cooijmans, Ciuca, Zhang, Gidel and Courville.
Authors
- Alejandro Panera Pascual (ORCID: https://orcid.org/0009-0008-5194-7399)
Institutions
- Universidad de Salamanca (ES)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24325532
- Primary Topic
- Game Theory and Applications
- Type
- preprint