Evaluating Transformer Architectures for Fault Detection in Industry 4.0: A Multivariate Time Series Classification Approach

The transformer architecture has revolutionized natural language processing (NLP), achieving unprecedented performance across several tasks. Transformer variants like BERT, GPT, and T5 have pushed the boundaries of language understanding and generation. This paper presents a comprehensive comparative analysis of state-of-the-art transformer architectures, with a novel focus on their applicability to multi-classification time-series problems. We examine structural innovations, training methodologies, and performance metrics, revealing each model’s strengths and limitations. Our analysis highlights the architectural designs of each model with their relative advantages and drawbacks, providing insights into their effectiveness for various applications. This work offers a valuable analysis of transformer models for optimal performance, particularly in multivariate time series analysis.