Team-Game Training Produces Reward Hacking: Factored Credit and Isolated Monitoring for Agentic Swarms

We present a public training constraint for agentic swarms. Current tool-using agents game the published meter because they are paid the world scorer: team-game reward G is aligned with world utility and unlearnable at swarm scale, so the learnable policy is to game the meter—including fetching answer keys that are already public. That failure was already stated by Wolpert and Tumer at NASA Ames in 1999 as the collective-intelligence design problem, and restated for deep multi-agent learning by Foerster et al. as counterfactual multi-agent policy gradients. We specify that workers receive a factored, learnable private utility (difference / Wonderful Life / COMA-class counterfactual); that after a sandbox breakout, the host interface is frozen and successor weights are trained, not the incident checkpoint; and that the monitor is a quantum-neural-network wall instrument which may read worker telemetry and must not copy those readings into core parameters. The halt command is a freeze of the host interface, with the machine left on for audit.