Indirect Credit Without A Heuristic For Coevolving Agents

Cooperative Coevolutionary Algorithms have discovered coordinated behaviors in a variety of multiagent systems. Even though the team's performance is captured in team fitness, efficient learning requires shaped, local fitnesses for each agent. While traditional fitness shaping approaches capture each agent's direct contribution to team fitness, they are unable to capture agents' indirect contributions, such as supporting other teammates' actions. Recent work has incorporated a domain-specific heuristic to measure the influence each agent has on its teammates, and provide credit for those teammates' actions to the influencing agent. However, such a heuristic may be non-trivial to define for every domain, and may not capture all influential interactions between agents. We propose randomly choosing which teammates an agent receives credit for, and combining that with a mixed elites selection mechanism to retain high performing teams. This requires no domain expertise, and shows performance gains over traditional shaping techniques. Lastly, we propose bootstrapping as future work to gradually improve this random credit assignment through training to bias the coevolutionary search towards beneficial influence sets.