HET-VQVAE: a novel encoder-decoder framework for irregularly-sampled multivariate time series of patients

Irregularly-sampled multivariate time series (IMTS) is ubiquitous in electronic health records (EHRs) and raises challenges to deep learning models to analyze it. In this study we propose a novel neural-network module named Hybrid Element-wise Transformer (HET), which uses a combined attention mechanism to integrate the sparse measurements, missing pattern and temporal irregularity in IMTS by element-wise matrix multiplication. We use the HET as the encoder and decoder within the Vector Quantised Variational Auto-Encoder (VQVAE) to build an encoder-decoder framework named HET-VQVAE, where the VQVAE discretizes the latent states to model IMTS. Our experiments on three ICU datasets show that HET-VQVAE has better or comparable performance compared to a range of routine or previously state-of-the-art baseline models on the tasks of interpolation, hospital mortality prediction and sepsis prediction, and meanwhile has high training efficiency. In addition, we use the t-Distributed Stochastic Neighbor Embedding (t-SNE) to visualize the latent states of different encoder-decoder frameworks and show that the discrete latent states of HET-VQVAE can precisely and robustly model IMTS. In conclusion, the HET-VQVAE is a promising deep learning framework to learn discrete latent states from IMTS for different clinical prediction tasks.