Granular Q-learning adaptation boosts collective welfare in multi-agent Prisoner’s Dilemma

Understanding how cooperation emerges and stabilizes in a difficult environment is a core challenge across biology, physics, and the social sciences. We present a reinforcement-learning framework for the Prisoner’s Dilemma Game between the two distinct agent types: Interactive Identity (II) and Interactive Diversity (ID). While II agents compress all neighbor interactions into one strategy update, ID agents assign one strategy to each neighbor, enabling finer-grained strategic adaptation. We systematically sweep dilemma strengths and analyze both homogeneous and heterogeneous network structures to show that ID agents persistently outcompete II agents at sustaining cooperation, especially for moderate temptations to defect. Moreover, in scenarios where agents can shift from II to ID based on relative payoffs, ID learning often invades populations of II learners, though influential hub nodes can impede this transition in heterogeneous networks. Spatiotemporal analyses indicate that ID agents form a strong cluster of cooperation, which prevents defection from spreading. Finally, extrapolating these results to wider moral dimensions, such as honesty, trust, and punishment, can give a rich understanding of how this granular, neighbor-specific learning raises collective welfare within both natural ecosystems and engineered multi-agent systems. • ID agents maintain higher cooperation than II agents on networks. • Switching from II to ID improves cooperation under moderate defection. • Spatiotemporal analysis shows ID agents form robust cooperative clusters.