A Comparative Evaluation of Synthetic Load Profile Generators Across Unseen Datasets

Datasets play crucial roles in successful network management. Mitigating the risk of congestion and faults, while offering a seamless energy transition, is enabled by complex, innovative systems fueled by high-quality data. However, in most cases, it is difficult to attain them, and even then, their use is limited due to concerns about privacy. Emerging synthetic data generators are trying to circumvent the issue; their effectiveness across unseen datasets is yet to be thoroughly investigated. Typically, there are not enough resources or emphasis to thoroughly evaluate the novel methodologies during the invention phase, despite the unprecedented advancement rate of artificial intelligence in recent years. This paper presents a comparative evaluation of synthetic smart meter generators on diverse public datasets to emphasise the importance of considering the background of training datasets and the evaluation of generalisation and applicability of the models across various cultural, geographical, and behavioural contexts. Results reveal that the accuracy of the models is dataset-dependent, yielding different outcomes across multiple datasets, albeit robust in other properties. The findings in the paper highlight the need for comprehensive benchmarking of synthetic data generators across diverse datasets. Moreover, the research underscores once more the key trade-offs between fidelity and privacy for future applications in smart grid analytics.