The Challenge of Generating and Evolving Real‐Life Like Synthetic Test Data Without Accessing Real‐World Raw Data—A Systematic Review

ABSTRACT Background High‐level system testing of applications that use data from e‐Government services as input requires test data that is real‐life‐like but where the privacy of personal information is guaranteed. Applications with such strong requirement include information exchange between countries, medicine, banking, and so on. This review aims to synthesise the current state‐of‐the‐practice in this domain. Objectives The objective of this Systematic Review is to identify existing approaches for creating and evolving synthetic test data without using real‐life raw data. Methods We followed well‐known methodologies for conducting systematic literature reviews, including the ones from Kitchenham and PRISMA as well as guidelines for analysing the limitations of our review and its threats to validity. Results A variety of methods and tools exist for creating privacy‐preserving test data. Our search found 1013 publications in IEEE Xplore, ACM Digital Library, and SCOPUS. We extracted data from 75 of those publications and identified 37 approaches that answer our research question partly. A common prerequisite for using these methods and tools is direct access to real‐life data for data anonymization or synthetic test data generation. Nine existing synthetic test data generation approaches were identified that were closest to answering our research question. Nevertheless, further work would be needed to add the ability to evolve synthetic test data to the existing approaches. Conclusions None of the publications covered our requirements completely, only partially. Synthetic test data evolution is a field that has not received much attention from researchers but needs to be explored in Digital Government Solutions, especially since new legal regulations are being put in force in many countries.