Effective Medical Visual Question Answering Using Dynamic Prompting and Decoding Knowledge Editing

Abstract This paper addresses the issue of time-consuming manual prompt design and the limitations of current knowledge editing methods, which fail to utilize new knowledge fully. It integrates knowledge during the pre-training and answer prediction stages. Additionally, it proposes a medical visual question-and-answer model called DPDE, which is based on dynamic prompts and decoded knowledge editing. This model innovatively applies dynamic prompts to the field of medical visual question answering. It proposes a new paradigm for decoding knowledge editing, which enhances the language model’s knowledge editing capabilities in the decoding stage. In order to verify the effectiveness of the DPDE model, we conducted multiple sets of experiments to explore the model performance on three data sets. Experiments have proven that the dynamic prompt module and decoding knowledge editing module used in this model can effectively improve the performance of medical visual question-answering tasks.