A Residual Network and Bi-directional LSTM based Hybrid Approach to Remote Sensing Image Captioning

Remote sensing imagery and captioning are invaluable when it comes to monitoring the Earth’s surface for different objectives such as environmental effects, land usage and monitoring like agriculture, disaster relief, etc. Based on the residual convolutional neural networks, ResNet50 and bi-directional long-short term memory (Bi-LSTM) model, this work proposes a hybrid approach to accomplish remote sensing image captioning. The comparison of the proposed approach is analyzed against different state-of-art CNN models and LSTM networks for image spatial and spectral feature learning in remote sensing imagery using Bleu scores, Meteor and Rouge metrics. The study provides an understanding of which kind of and how these systems should be developed to facilitate better data analysis and usage to enhance the construction of elaborated remote sensing image captioning framework to support specific image interpretation for the prevention of environmental degradation, disasters, etc. The proposed ResNet50-BiLSTM model is the most effective in terms of achieving the highest BLEU-1 with 75.45%, BLEU-2 with 65.98%, BLEU-3 with 56.54%, BLEU-4 of 44.636%, ROUGE with 42.65% and METEOR with 40.69% scores, and higher generality than other models.