The Specificity vs. Expense Trade-Off of Multiagent Credit

Many real-world settings call for multiple autonomous agents to coordinate and work as a collective. In such settings, cooperative coevolution—which trains a policy for each agent in parallel—has emerged as a popular choice. However, providing these coevolving agents only with feedback that evaluates the whole system of agents is often suboptimal, as learning is each individual agent's individual responsibility. This necessitates deriving agent-specific feedback from the system feedback, termed the "multiagent credit assignment problem". Oftentimes, however, such isolation of agent-specific feedback may be computationally expensive, and may not yield commensurate improvements in learning efficiency. In this paper, we present Dflex, an extension of Difference Evaluation that offers tuneable specificity in the feedback computed for each agent. This tuneability—achieved by allowing a subset of coevolving agents to 'share' feedback—allows practitioners to trade between feedback specificity and its compute expense. Our initial results in the multiagent rover exploration problem not only empirically indicate the presence of the feedback specificity vs. compute cost trade-off, but also show Dflex's ability to provide control over it.