Adaptive CNN-Transformer Fusion for Discriminative Fuzzy $C$-Means Clustering in Multivariate Forecasting

With the rapid advancement of artificial intelligence, an increasing number of researchers have begun to integrate deep learning with machine learning to propose a variety of meaningful prediction models. However, determining the optimal number of convolutional and pooling layers in a convolutional neural network (CNN) remains a challenging problem. Additionally, the randomness in the query, key, and value vectors of the self-attention mechanism in Transformers can affect their predictive performance. Furthermore, inaccuracies in the calculation of the membership matrix in fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$C$</tex-math></inline-formula>-means clustering (FCM) can impact clustering effectiveness. To address these issues, this paper introduces an adaptive CNN-Transformer multivariate prediction model based on discriminative fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$C$</tex-math></inline-formula>-means clustering (DFCM), referred to as DFCM-A-CNN-Transformer. Firstly, by considering the membership degree of similar samples belonging to the same cluster, DFCM is designed to resolve inaccuracies in the computation of the membership degree matrix. Next, a system is developed to simultaneously determine the value of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> and the number of clusters in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula>-nearest neighbor algorithm, thereby completing feature selection. Subsequently, the number of convolutional and pooling layers is adaptively determined based on data characteristics and dynamic stopping conditions. The query, key, and value vectors of the self-attention mechanism in the Transformer are then determined based on the output of the CNN. Finally, the superiority and effectiveness of the proposed model are validated through comparative experiments with six other models on eight datasets.