Context-Aware Cooperative Gating for Knowledge-Controlled Value Factorization in Multi-Agent Reinforcement Learning

Cooperative multi-agent reinforcement learning under partial observability requires decentralized agents to act from incomplete local information. Value-decomposition methods such as VDN and QMIX do not control when teammate-related knowledge may influence an agent’s local utility, and an uncontrolled teammate-conditioned term can distort decisions at timesteps where no ally is observable. We propose CA-QMIX, a context-aware value factorization method that uses ally visibility as a knowledge signal for decentralized agent control. Each agent combines an individual decision branch with an attention-based cooperative knowledge branch through a visibility-conditioned gate that scales the cooperative contribution and is hard-masked to zero when no ally is observable. The gated utilities are combined by a QMIX-style monotonic mixer, which we prove preserves Individual-Global-Max consistency, and a mixer-level Δ-Q regularizer discourages cooperative corrections that lower the greedy value of the base branch. Experiments on seven SMAC v1 and SMAC v2 scenarios against six value-decomposition baselines show that CA-QMIX attains the highest mean final win rate and the best mean rank over the seven scenarios and improves on QMIX in all three randomized SMAC v2 scenarios. Ablations show that most of the gain comes from the cooperative branch, that its visibility conditioning helps when correctly aligned (misaligned visibility degrades performance under randomized placement), and that the form of the gate is not critical: a binary gate suffices and the hard no-ally mask has no measurable cost.