Judge: Effective State Abstraction for Guiding Automated Web GUI Testing

Automated web GUI testing approaches aim to maximize the code coverage of a web app within a specific time budget. However, due to the highly dynamic characteristics of web apps, testing approaches often get stuck in loops or repeatedly explore the same app areas. To address this issue, existing approaches conduct state abstraction, grouping similar pages into the same state in an effort to approximate the ideal state (i.e., a state that encompasses all-and-only those pages exhibiting the same behavior from a testing perspective) to reduce repetitive explorations. Typically, these approaches rely on the Document Object Model (DOM) or visual similarity, using predefined thresholds or learning-based classifiers to determine which pages should belong to the same state. However, pages within the same ideal state still exhibit discrepancies, caused by factors such as dynamically loaded data and dynamically expanded UI elements. The varying page complexities and design styles among apps bring even more challenges. These phenomena present substantial obstacles to existing approaches in determining desirable classification thresholds or training desirable classifiers, preventing them from conducting satisfactory state abstraction to guide the testing process. To address the preceding challenges, in this article, we propose Judge, a novel approach based on structure merging and contrastive learning for state abstraction. Judge includes a “merge-and-classify” strategy. In the “merge” phase, Judge iterates through the DOM tree of each given page and merges web element siblings that share the same subtree structure into a single one to abstract and simplify the page, while discarding text contents and HTML attributes of web elements in the process. In this way, Judge mitigates the negative effects introduced by dynamically loaded data and dynamically expanded UI elements, substantially reducing discrepancies between pages in the same ideal state. In the “classify” phase, Judge uses a dedicated contrastive learning model to embed simplified page DOMs into vectors and further conducts classification with a Support Vector Machine (SVM), enabling classification in high-dimensional vector space and improving generalizability across diverse web apps. We evaluate Judge against 13 widely used baseline approaches. The results highlight that Judge outperforms these baseline approaches in classifying page pairs, with an average margin ranging from 8.95% to 28.90% in the F1 score across three manually labeled datasets. Additionally, when compared to the five most effective baseline approaches, Judge demonstrates superiority in guiding the exploration of automated web GUI testing in six widely studied apps, with code coverage improved by an average of 2.62–14.12%. The code and data of Judge are publicly accessible.