A Method for Optimizing Electric Power Vision Model Based on Network Architecture Search and Attention Mechanism

The attention mechanism is widely used in large language models and multimodal models. How to apply the attention mechanism to a single visual field is a question worth considering. This article provides an power visual model optimization method based on attention mechanism. Unlike Vi<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex>, Vi<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex> does not use convolution operations, but divides the image into small patches and directly processes global features through the Transformer model. The method proposed in this article is based on the existing deep neural network using convolution. It does not abandon the original structure and is used to solve the problem that the performance of the original model is insufficient and needs to be improved. However, blindly using the attention mechanism optimization will lead to overfitting, and the performance may decline instead. In addition, the existing network architecture search (NAS) algorithm cannot be fully and directly applied to the search for appropriate attention mechanisms.