MFSL-CGA: a multimodal few-shot learning method for time series classification with cross-class guided attention

Time series classification can exploit the feature patterns of temporal data, which is essential for decision-making and risk control in industry, healthcare, and finance. However, most real-world application scenarios face the problem of complex feature patterns of time-series data and the scarcity of labeled data, which makes the time-series classification problem in few-shot scenarios challenging in the present research. To solve the above problems, we propose a multimodal few-shot time series classification method based on a Cross-Class Guided Attention mechanism, called MFSL-CGA. MFSL-CGA first constructs a multimodal feature extraction module, which utilizes the Gramian Angular Summation Field image method for modal expansion to encode the time series data into images. The two modalities of time series and images complement each other to jointly encode temporal dynamics and pairwise spatial correlations, thereby improving class separability in the feature space. Then, we build a dual-branch feature extraction and fusion network to mine the multimodal data’s local and global temporal relationships and obtain highly distinguishable fusion features. In addition, we propose a Cross-Class Guided Attention (CGA) mechanism, which explicitly guides the attention mechanism in mining the associative relationships of sample categories. CGA uses training labels during the task-based training phase to improve the discriminability between sample categories. In turn, the prototypes of sample categories are computed by a nearest-neighbor prototype classifier to achieve classification. Finally, we perform extensive comparative experiments on the UCR standard time series datasets. Experimental results demonstrate that the MFSL-CGA achieves average accuracy of 86.89%, 78.78%, 84.57%, and 79.79% on the 5-way 5-shot, 5-way 1-shot, 3-way 5-shot, and 3-way 1-shot tasks, respectively, achieving optimal classification performance across more datasets. Additionally, the proposed method achieves an average accuracy of 68.86% in cross-domain few-shot time series classification and demonstrates optimal classification accuracy across 13 datasets.