Accelerating Multi-Agent DDPG Training on Multi-GPU Platforms

Multi-Agent Reinforcement Learning (MARL) is a crucial technology in artificial intelligence applications. Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is a state-of-the-art MARL algorithm that has gained widespread adoption and is considered a popular baseline for comparing against novel MARL algorithms. The training process of MADDPG systems involves a high volume of data communication and computation when scaling to larger problem sizes. While state-of-the-art CPU-GPU systems provide the necessary computing power for high-throughput training, they do not efficiently utilize underlying platform resources and are unable to facilitate training using multiple GPU devices. We propose a novel accelerated system for MADDPG training that leverages multiple GPU devices while additionally increasing the utilization of CPU resources. We access our MADDPG system on multiple benchmarks on a multi-GPU platform, resulting in up to a 1.5× higher system throughput compared to state-of-the-art CPU-GPU systems.