Improving robustness of Transformers for power quality disturbance classification via optimized relevance maps

Power quality disturbances (PQDs) are a critical area of research in the power systems domain. The growing complexity of modern power systems increases the variability of PQDs that may occur, making it challenging to manage power quality using traditional methods, and motivates the inclusion of machine learning models. However, these models are vulnerable to adversarial attacks. The predominant defense approach is adversarial training, which increases robustness at the price of significant increase in computational complexity of the training process, and can often lead to reduced accuracy on non-adversarial examples. In this light, this paper introduces a distinct approach to increase the robustness of Transformer-based PQD classifiers by leveraging their intrinsic explainability derived from the self-attention mechanism. By incorporating the resulting explainable features in the form of relevance maps into the loss function during the fine-tuning stage, we guide the model to focus on salient disturbance features. This method enhances the robustness against adversarial attacks, without compromising accuracy, and avoids the additional computational overhead associated with adversarial training. Experimental results show that when no adversarial attack is applied, the fine-tuning stage increases the PQD classification accuracy from 97.62% to 99.26%, thus overcoming the accuracy-robustness trade-off. Furthermore, the Transformer with the proposed fine-tuning stage is more adversarial robust than a state-of-the-art deep Convolutional neural network (DeepCNN) PQD classifier. The proposed Transformer maintains accuracy and F1 scores slightly above 88%, whereas fo DeepCNN both scores are slightly below 66.60%, thus highlighting the effectiveness of our approach.