Time Series Classification with Language Models

Time series classification is a crucial yet challenging task due to the diverse and complex nature of time series data. While deep learning methods, particularly Convolutional Neural Networks (CNNs), have been widely explored for this problem, transformer-based approaches remain relatively underutilized. With the rise of large language models, recent research has begun to investigate how various data types, including time series, can be integrated into language models. In this paper, we explore the transformation of time series data into symbolic representations for use with language models such as BERT. Symbolizing time series data is a foundational and non-trivial step, as the choice of symbolization method can significantly affect downstream performance. We examine three different symbolization techniques and analyze their impact on classification accuracy. Additionally, we investigate the effect of task-adaptive pretraining using an expanded symbolic time series corpus. By combining these elements, our study provides a analysis of applying language models to time series classification, contributing valuable insights into the effective utilization of symbolic representations and pretrained transformers in this domain.