Enhancing Learning to Communicate With Reward-Shaped Curriculum and Network Awareness

Communication enhances collaboration among artificial intelligence agents, for example, by sharing observations that contribute to safer driving. Given the conflicts between limited communication resources and communication needs, learning effective communication strategies is essential. We observe that incorporating learning to communicate can complicate mastering primary tasks, like vehicle control, the original focus in autonomous driving. This is due to the uncertainty in information acquisition during the learning process, which can lead to an unstable environment for primary tasks. In this paper, we introduce ReSCOM, an efficient joint learning framework that combines learning-to-communicate with primary tasks. ReSCOM progressively adjusts the learning emphasis through rewardshaped curriculum, allowing agents to shift their focus from primary tasks and basic communication tasks (e.g., how to encode) to advanced communication strategies (e.g., determining when it is worthwhile to communicate). This approach minimizes the impact on the learning efficiency of primary tasks while simultaneously facilitating communication learning. Besides, we explore the extent to which communication channel states (i.e., delays and packet loss) and protocols impact agent cooperation and learning. We evaluate ReSCOM against state-of-the-art methods in various tasks, demonstrating its strong performance. Furthermore, we verify that current modern wireless channels, includingWi-Fi, 4G, and 5G, provide low enough delays that their impact can be ignored. When packet loss occurs, we find that the UDP protocol performs better than TCP because, for agent cooperation, timely information is more valuable than reliability.