MedVQA-APL: enhancing medical visual question answering with assistance prompt learning

In order to assist physicians in reducing repetitious chores and lightening their workload, Medical Visual Question Answering (Med-VQA) intends to streamline the process of extracting accurate responses from medical images and related queries. Existing medical visual question answering models are frequently pre-trained on extensive medical datasets before being fine-tuned on smaller, task-specific datasets. Despite the increased parameterization associated with these models, the enhancement in performance is often marginal. We present Medical Visual Question Answering with Assistance Prompt Learning (MedVQA-APL) in this study. It uses an encoder that is rather simple and still has strong performance. Our method starts with the creation of an assistance prompt generator module whose purpose is to generate an assistance prompt that is limited by the intended response. Next, we introduce an assistance prompt fusion module incorporated within a multi-modal fusion block. This module leverages the assistance prompt to derive clinically relevant features. In order to forecast the final answers, the integrated information is finally merged with image-language cross-modal information. Two publicly accessible Med-VQA datasets were used for experimental assessments, which show that MedVQA-APL outperforms the conventional model and produces performance improvements of 2.85% and 5.54% on VQA-RAD and Path-VQA, respectively.