Poster Abstract: Exploring the Autoencoder Sequence Pooling

Sequence embeddings are essential for tasks like time series analysis and natural language processing, yet pooling techniques to create compact sequence representations remain underexplored. Poor pooling methods can lead to significant information loss, diminishing the effectiveness of strong feature extractors. In this early results paper, we propose an autoencoder-based sequence pooling approach that leverages autoencoders' ability to compress information and supports pretraining during self-supervised learning. Evaluated on time series data with a transformer-based encoder, our method outperforms traditional pooling techniques, such as mathematical functions and learnable weighted sums.