Dataset generation and model fine-tuning for GUI anomaly detection

In the rapidly evolving smartphone industry, ensuring the quality and usability of mobile applications is crucial, particularly through robust Graphical User Interfaces. Traditional manual GUI testing methods no longer support the complexity of modern applications. This paper presents a methodology for GUI anomaly detection leveraging synthetic data generation and deep learning. Our approach addresses the limitations of existing methods by creating a dataset that accurately replicates real-world GUI anomalies, such as cropped text, overlapping elements, pixelation, and null values. We used the dataset generated to train and evaluate the well-known YOLOv8 model, attaining 97% mAP to recognize the anomalies. Our study underscores the potential of automated, scalable solutions for automated enhancing GUI testing.