Real-time forecasting of chaotic dynamics from sparse data and autoencoders

The real-time prediction of chaotic systems requires a nonlinear-reduced order model (ROM) to forecast the dynamics, and a stream of data from sensors to update the ROM. Data-driven ROMs are typically built with a two-step strategy: data compression onto a lower-dimensional latent space, and prediction of the temporal dynamics on it. Although these methods have proven effective, to achieve real-time prediction, there are two key challenges to overcome: (i) ROMs of chaotic systems can become numerically unstable; and (ii) sensors’ data are sparse, i.e., partial, and noisy. To overcome these challenges, we propose an augmented sequential data assimilation (DA) framework based on the Ensemble Kalman filter (EnKF) that updates the latent state of ROM by assimilating noisy and sparse measurements. We demonstrate the proposed DA-ROM framework using a ROM that consists of a convolutional autoencoder (CAE) to compresses the system’s state onto a lower-dimensional latent space, and an echo state network (ESN) formulated as a state-space model to forecast the temporal evolution on the latent space. The DA-CAE-ESN provides a numerically stable and real-time adaptive ROM. The DA-CAE-ESN is tested on spatio-temporally chaotic partial differential equations: the Kuramoto–Sivashinsky equation, and a two-dimensional Navier-Stokes equation (Kolmogorov flow). We show that the method provides accurate and stable forecasts across different levels of noise, sparsity, and sampling rates. As a by-product, the DA-CAE-ESN acts as a localization strategy that mitigates spurious correlations, which arise when applying the EnKF to high-dimensional systems. The DA-CAE-ESN provides a numerically stable method to perform real-time predictions, which opens opportunities for deploying data-driven latent models.