Evaluating the Suitability of Inception Score and Fréchet Inception Distance as Metrics for Quality and Diversity in Image Generation

Variational Autoencoders (VAEs) have gained popularity as one of the main approaches for generating diverse and high-quality synthetic images. This study examines the suitability of evaluation metrics, specifically Inception Score and Fréchet Inception Distance (FID), for assessing these images. Particularly, the study focuses on the generation of synthetic images based on the MNIST handwritten digits dataset. Through the use of VAE-generated MNIST image samples, the study analyses the abovementioned metrics alongside alternative methods that can be used to assess image quality and diversity. The findings made from the study reveal the strengths and limitations of each metric in evaluating image quality and diversity. This paper underscores the need for tailored metrics to enhance the evaluation of generative models, while specifically using the performance of a VAE as the domain of investigation.