MDCSformer: Multi-scale dilated contracted sub-transformer for fault diagnosis of rotating machinery

Abstract Transformers have achieved remarkable success in natural language processing and computer vision, motivating their extensive research into intelligent fault diagnosis. The multi-head self-attention mechanism in Transformers effectively extracts global features and emphasizes important feature maps. However, directly applying Transformers to vibration signals can incur substantial computation and training effort, which often makes convolutional neural network (CNN)-based approaches more practical. This paper, thus, proposes a new architecture, called Multi-scale Dilated Contracted sub-Transformer (MDCSformer), specifically designed to efficiently process noisy vibration signals for fault diagnosis. First, multi-scale dilated convolution layers capture features at various scales, extracting both local and global characteristics from the signals. Next, a contracted multi-head sub-self-attention module refines these features by selectively emphasizing relevant information while maintaining computational cost. The proposed method is validated with two gearbox vibration datasets and a bearing vibration dataset, demonstrating that MDCSformer outperforms comparable CNN-based, Transformer-based, and hybrid models by achieving higher accuracy and robustness in fault diagnosis. This superior performance stems from its integration of rich multi-scale feature extraction with contracted subspace attention.