When Local Credit Assignment Breaks: Exact Diagnostics of Additive and Pairwise Reward Projection
Abstract Average reward reconstruction error is an incomplete measure of the decision quality of a factored cooperative model. We provide a small, fully reproducible diagnostic of this established issue, rather than a new multi-agent reinforcement learning algorithm. In a binary one-step game with a rare joint synergy, the exact uniform least-squares additive projection has mean squared error 2⁻ⁿ − (n + 1)4⁻ⁿ but coordination regret 1 − 3n2⁻ⁿ. Adding any set of pairwise factors reduces reconstruction error without changing the wrong maximizing action. Exhaustive evaluation at up to 16 agents confirms the formulas; 800 independent sampled fits illustrate that observing the synergistic action need not repair the population projection objective. A privileged twofold weight on the known optimal action reverses the policy while increasing uniform reconstruction error. At 16 agents, additive MSE is 1.52548 × 10⁻⁵ and regret is 0.999268. However, MSE divided by reward variance is 0.999939, exposing the apparently small absolute error as a sparse-reward averaging effect. The package supplies proofs, controlled ablations, an established matrix-game check, raw results, and tests. It does not reproduce LOMAQ or establish a new failure of neural value-decomposition algorithms.
Authors
- Krishna Varun Kasamneni
Publication Details
- Journal
- Research Square
- Published
- 2026-11-10
- DOI
- https://doi.org/10.21203/rs.3.rs-11657377/v1
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- preprint