On the prevalence of test smells in mobile development

Objective: This study explores the quality of tests in Dart, the main language for mobile application development with the Flutter framework. Methods: The study begins by using the DNose tool, used to detect 14 types of test smells in code written in the Dart language. Next, we evaluate the tool’s precision, accuracy, recall, and F1-score. Using this tool, we conduct a detailed analysis of tests in open-source projects extracted from the language’s central repository. Results: The study starts with a dataset of 5,410 Dart-language projects, from which we were able to clone 4,154 repositories after processing. Based on the cloned projects, we generated a dataset containing 907,566 occurrences of test smells. Through our analysis, we characterized the specific types of test smells most frequently encountered and identified their causes. We observed the presence of test smells in 74% of test files. Another noticeable characteristic among the analyzed projects was the scarcity of tests, with 1,873 projects having one or no tests, which led us to expand the number of analyzed projects to a broader base. Conclusion: This research makes a significant contribution by providing insights into the quality of tests in projects from Dart’s official repository, as well as by offering an open-source tool for detecting 14 types of test smells.