Lightweight method for cognitive large model of electric power equipment operation and inspection based on knowledge distillation

The artificial intelligence big model has been successfully applied in the maintenance of power equipment, but power equipment maintenance involves multiple modalities of data. The high computational complexity and storage requirements make it difficult to widely promote under limited conditions. In response to this issue, a model compression method based on knowledge distillation is proposed in the paper to simplify the artificial intelligence cognitive large model for power equipment operation and inspection. The operation and inspection data of power equipment generally have complex data or knowledge backgrounds, and the target knowledge in the knowledge base is relatively small. To avoid data noise, the updated data from the previous iteration is used to guide the training of the model for the next iteration. Separate the feature information of the teacher model into local and global features and train the student model separately by calculating local and global losses. Introducing knowledge adjustment mechanisms and dynamic temperature distillation mechanisms, allowing student models to autonomously select the core features of images, in order to reduce the misleading effects of incorrect predictions in the model. Finally, the application is combined with the operation and inspection data of power equipment to verify the effectiveness of the model lightweight method described in the paper.