Continual Learning for Image Captioning in Hindi

Continual learning, alternatively referred to as incremental learning or lifelong learning, represents a learning paradigm enabling an agent to acquire new information without compromising its retention of previously learned knowledge. Continual learning research has resulted in a number of ways to prevent catastrophic forgetting in deep neural networks. Apparently, continuous learning of recurrent models has never been used to solve the image captioning problems in the Hindi language. This research presents a novel approach to employ continual learning in the context of generating image captions in Hindi. Given its widespread usage in South Asia and its official status in India, Hindi ranks as the third most spoken language worldwide. We look at the continual learning of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) based models for image captioning in a systematic way. In continuous image captioning problems, we incorporate an attention-based method that explicitly addresses the transient aspect of vocabulary. Using MS-COCO datasets, we employ our methods to solve the incremental image captioning problem. Our findings show that the proposed method can learn five captioning tasks in consecutive order without forgetting the one it has already learned.