Comparison of Various Vision-Language Models: Datasets, Architectures, And Metrics

We aim to provide a thorough comparison of different vision-language models for the picture captioning task, on several datasets, such as COCO, Flickr8k, and MS COCO. In particular, we analyse the CNN-RNN, CNN-Transformer, Vision Transformer (ViT) + Transformer Decoder, Convolutional Sequenceto Sequence (ConvS2S), and Cross-modal Transformers (like Oscar) Encoder-Decoder architectures. In future work, we intend to train and test these models on a range of datasets in order to assess their potential accuracy and effectiveness in generating descriptive captions. Important metrics including BLEU, METEOR, and CIDEr scores could be used in the evaluation process to provide light on the advantages and disadvantages of each model-dataset combination. The results of this research are meant to direct the choice of ideal structures for particular image captioning jobs, advancing personalized and adaptive image description systems, especially for visually challenged users.