Fine-Tuning of the Power Distribution System Inspection VQA Model Based on Box Annotation

Large Models (LMs) have achieved excellent results in general domains, and their application in vertical domains has been a research hotspot. This paper attempts to convert box annotations to Visual Question Answering (VQA) labels for LM's fine-tuning in the distribution inspection domain. The proposed approach can effectively utilize the cheap and easily accessible box annotations, predefined question templates, power equipment Knowledge Graph (KG), and randomization method for VQA label generation. This paper chose the BLIP model for fine-tuning and evaluation, and the results show that the BLIP can understand the content of distribution inspection images better and generalization in multiple questions. However, this paper also points out that the application of the LM in the field of inspection may still have problems such as poor cost-effectiveness and loss of generalization ability.