Can Cooperative Multi-Agent Reinforcement Learning Boost Automatic Web Testing? An Exploratory Study

Reinforcement learning (RL)-based web GUI testing techniques have attracted significant attention in both academia and industry due to their ability to facilitate automatic and intelligent exploration of websites under test. Yet, the existing approaches that leverage a single RL agent often struggle to comprehensively explore the vast state space of large-scale websites with complex structures and dynamic content. Observing this phenomenon and recognizing the benefit of multiple agents, we explore the use of Multi-Agent RL (MARL) algorithms for automatic web GUI testing, aiming to improve test efficiency and coverage. However, how to share information among different agents to avoid redundant actions and achieve effective cooperation is a non-trivial problem. To address the challenge, we propose the first MARL-based web GUI testing system, MARG, which coordinates multiple testing agents to efficiently explore a website under test. To share testing experience among different agents, we have designed two data sharing schemes: one centralized scheme with a shared Q-table to facilitate efficient communication, and another distributed scheme with data exchange to decrease the overhead of maintaining Q-tables. We have evaluated MARG on nine popular real-world websites. When configuring with five agents, MARG achieves an average increase of 4.34 and 3.89 times in the number of explored states, as well as a corresponding increase of 4.03 and 3.76 times in the number of detected failures, respectively, when compared to two state-of-the-art approaches. Additionally, compared to independently running the same number of agents, MARG can explore 36.42% more unique web states. These results demonstrate the usefulness of MARL in enhancing the efficiency and performance of web GUI testing tasks.