Scaling Regime-Aware Forecasting: Distributed Shifting Seasonal Matrix Factorization

Forecasting methods ingest time series data and provide a prediction of future events. One sophisticated forecasting algorithm is called the Shifting Seasonal Matrix Factorization (SSMF) method, which is notable for its ability to model repeating patterns in data as well as abrupt global changes in system behavior (regime shifts). SSMF as originally implemented does not scale well with input data size. To address this issue, we provide an MPI parallelization scheme with 2D matrix partitioning, sparsity-awareness, and better regime lifecycle management. We demonstrate the improved scaling and performance using NYC taxicab and rideshare data.