AI-Enhanced Test Automation Tool for Desktop Applications

Test automation is an essential part of the software testing process. This study aims to develop an AI-enhanced test automation tool for testing the user interfaces of desktop applications. The detection of each object in the graphical user interfaces of the software will be carried out using the object detection capabilities of YOLOv9 and Faster R-CNN models.The study emphasizes the importance of preprocessing steps for achieving successful outcomes in object detection processes. These preprocessing steps include image resizing, data augmentation techniques, and balancing the dataset. Additionally, the correct selection and optimization of hyperparameters (e.g., learning rate, number of epochs, network depth, and anchor box dimensions) in object detection models play a critical role in improving model performance.In this study, data analysis techniques using Python were utilized for hyperparameter optimization. Hyperparameters were evaluated and optimized based on metrics such as model accuracy, loss curves, and training time. As a result, high performance was achieved for both the test automation tool and the object detection process. This approach demonstrates the power of artificial intelligence and data analytics in test automation processes, serving as a significant example for both educational and practical applications.