A Non-Parametric Bayesian Approach Towards OnlineSequence Learning

Forecasting the next sample of an incoming sequence of noisy observations is a recurring problem in several real-world applications. In general, the streamed observations may originate from a possibly non-stationary stochastic process with non-linear dynamics. Very often though, to avoid devising explicit probabilistic models from scratch, we alternatively learn data-driven surrogate models from past observations to approximate complex dynamics. Time-varying systems in turn pose additional hurdles as such models must be continuously updated to accommodate changes in their underlying behavior. In this paper, we introduce a novel grid-based filter designed to inherently deal with the continuous learning problem. Specifically, Bayesian-grounded procedures are employed to both recursively predict observations and incrementally build a suitable non-parametric, probabilistic model of the indirectly observed phenomenon. Moreover, we also present a one-shot learning, memory-based implementation of the proposed filter, a shallow weightless neural network which is able to efficiently store and retrieve associative inputoutput pairs. Lastly, we demonstrate the effectiveness of our filter in performing simple anomaly detection tasks.