A Prompt That Can Stimulate the Self-Correction Ability of the VLM, Improving the Performance on VQA Task

Visual language Models (VLMS) are revolutionizing multimodal understanding by bridging the gap between vision and language, with great potential in diverse applications. How to improve its performance, so that it can better complete a variety of tasks has become the goal of researchers. To improve the performance of VLM on Visual Question answering (VQA) tasks, this paper proposes an innovative method, Self-correction Prompt, which integrates self-correction into prompt engineering by looping questions to improve accuracy while avoiding additional model training. Experiments on three typical VLMs show that Self-correction Prompt is effective, and the accuracy can be improved by 1.7% at most. It can also stimulate the self-detection ability of the model to find the previously made errors, and its efficiency can reach an average of 72%. The paper also discusses the types of tasks VLMs are not good at, which often prevent models from further improving their accuracy. The proposed method is simple and can be seamlessly integrated into a variety of VLMs, which provides a new idea for the following research on prompt engineering.