Nature-based Bengali Picture Captioning using Global Attention with GRU

Automatic picture captioning is a prominent re-search area of artificial intelligence technology (AI). Its ability to enhance AI models by translating observed data into human language opens up a wide range of real-time applications. In this study, we explore picture captioning in the Bengali language using the global attention mechanism. Given the limited prior research in this area, we provide a comprehensive assessment of two distinct global attention approaches: the general approach and the concatenation approach. Additionally, we evaluate the performance of two CNN encoders, VGG19 and InceptionV3,within these models. The models are trained on a secondary dataset consisting of 4,849 nature-based images, each annotated with a single caption, enabling the models to gain a broad understanding of related categorical information. To achieve our research objectives, we developed and trained four separate models using this new dataset. Our analysis, both qualitative and quantitative, demonstrates that these algorithms are capable of generating human-like captions for similar images. The results indicate that models using the concatenation approach, particu-larly with the InceptionV3encoder, performed best, achieving a BLEU-1 score of 84.85. In contrast, the model using the general approach with VGG19 underperformed in generating satisfactory captions.

Authors

Institutions

Publication Details

Published
2024-09-22
DOI
https://doi.org/10.1109/mlsp58920.2024.10734813
Citations
1
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.25
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Nature-based Bengali Picture Captioning using Global Attention with GRU

Fatema Tuz Zohora, Sujit K. Biswas, Anupam Kumar Bairagi, Kashif Sharif
1 citations
Multimodal Machine Learning Applications
0.25
article

Nature-based Bengali Picture Captioning using Global Attention with GRU

Fatema Tuz Zohora, Sujit K. Biswas, Anupam Kumar Bairagi, Kashif Sharif
article en
1 citations

Abstract

Automatic picture captioning is a prominent re-search area of artificial intelligence technology (AI). Its ability to enhance AI models by translating observed data into human language opens up a wide range of real-time applications. In this study, we explore picture captioning in the Bengali language using the global attention mechanism. Given the limited prior research in this area, we provide a comprehensive assessment of two distinct global attention approaches: the general approach and the concatenation approach. Additionally, we evaluate the performance of two CNN encoders, VGG19 and InceptionV3,within these models. The models are trained on a secondary dataset consisting of 4,849 nature-based images, each annotated with a single caption, enabling the models to gain a broad understanding of related categorical information. To achieve our research objectives, we developed and trained four separate models using this new dataset. Our analysis, both qualitative and quantitative, demonstrates that these algorithms are capable of generating human-like captions for similar images. The results indicate that models using the concatenation approach, particu-larly with the InceptionV3encoder, performed best, achieving a BLEU-1 score of 84.85. In contrast, the model using the general approach with VGG19 underperformed in generating satisfactory captions.

Khulna University (BD), Beijing Institute of Technology (CN), City, University of London (GB), Northumbria University (GB)
Climate action
Openalex Percentile: Top 45%
Multimodal Machine Learning Applications
0.25
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.