Enhancing Image Captioning Model Performance through use of Synthetic Image Generation and Semantic Filtering

The use of generative models to create synthetic data, which could be utilized to enhance datasets, is gaining a lot of interest recently. Although synthetic data can be used to increase the volume of data, there is a huge concern regarding the reliability of synthetic data created due to semantic inconsistencies, deviations from reality, and domain gaps, which can affect the performance of the model. In this research paper, we investigate the application of semantic filtering using cross-modal projection as a solution to these challenges. By training and comparing the accuracy of the image captioning model on three different datasets- dataset without synthetic data, dataset combined with unfiltered synthetic data, and dataset combined with semantically filtered synthetic data - we demonstrate the effectiveness of the proposed methodology for leveraging synthetic data for model training.