Metrics for Privacy-Preserving Generative Models: A Comprehensive Survey

Generative models are extremely widely used, both in research and practice. Although many studies have shown privacy threats against generative models or proposed privacy protections, there is no systematic survey on which metrics should be used to quantify privacy and utility of generative models. In this article, we therefore present new taxonomies for privacy and utility metrics for generative models, including taxonomies for attacks against generative models and a discussion of the most common approaches for privacy protection. We also propose guidelines for selecting appropriate metrics in specific scenarios and show commonly used metrics in several use cases. Finally, we discuss open challenges and promising future research directions.