CNN-Based Image Analysis for Detecting UI Defects in Mobile Apps: Advancing Automated GUI Testing with Generative Adversarial Networks

Automated detection of user interface (UI) defects in mobile applications has become one of the key elements in ensuring good user experience on multiple devices and screen resolutions. Particularly, traditional methods of manual testing are usually time-consuming and error-prone, making them unfit for large-scale testing. In this work, we propose a deep learning framework for automated graphical user interface (GUI) testing that combines convolutional neural networks (CNNs) and generative adversarial networks (GANs). The CNNs extract meaningful visual features from UI screen images, whereas GANs generate some synthetic data to solve the problem of the limited number of labelled samples and also to classify the UI images as defect or non-defect. The system was trained and tested on a dataset of mobile UI screenshots with a classification performance of 98.12% accuracy and 97.62% precision, indicating effectiveness and robustness of the approach in finding subtle and complex UI defects and thus enabling fast and scalable mobile app testing.