Incremental Image Captioning with Attention to Transient via Kolmogorov-Arnold Network

In this work, we present an incremental learning framework for image captioning, enabling models to learn new tasks across various image domains. We introduce the integration of Kolmogorov-Arnold Networks (KAN) with attention-enhanced long short-term memory (LSTM), mapping output features to vocabulary probability distributions for next-word prediction, termed "RATT-KAN". The encoder employs a contrastive continual learning loss for flexible adaptation to new tasks. Furthermore, within the decoder, under the task-specific Recurrent Attention to Transient Tasks (RATT) attention parameters, we incorporate the KAN to replace the traditional linear classifier. By utilizing the β-spline function provides greater flexibility and stability, thereby mitigating forgetting. We evaluated the image captioning task on the Microsoft Common Objects in Context (MS-COCO) and Flickr30K datasets. The experimental results demonstrate the superior performance of our newly developed comprehensive model.