Meta-LSTM-Affine: A Memory-Based Meta-Adaptive Affine Modeling Framework for Non-Stationary Systems

Modeling non-stationary systems with dynamically evolving data distributions remains a fundamental challenge in modern learning and optimization problems. In this work, we adopt a generalized notion of non-stationarity, where distribution shifts across tasks and domains are treated as forms of non-stationary processes. This perspective allows us to study non-stationary behavior in controlled settings such as Few-Shot Learning (FSL) and Source-Free Domain Adaptation (SFDA), where data distributions vary across episodes or domains. Conventional normalization and feature modulation strategies often rely on batch-level statistics, leading to unstable behavior under small-batch, streaming, and distribution-shifted conditions. To address these limitations, we propose Meta-LSTM-Affine, a memory-based meta-adaptive affine modeling (normalization) framework that unifies recurrent temporal memory and meta-learning for robust feature modulation. Unlike batch-statistics-driven normalization, our method employs an LSTM-based affine parameter generator (APG) to dynamically produce channel-wise scale and shift parameters based on both current inputs and historical context. To further enhance task-level adaptability, we introduce three lightweight meta-learning mechanisms—Meta-Initialization, Meta-Conditioning, and Meta-Update—that enable rapid cross-task adaptation without modifying the backbone. A bi-level training strategy with temporal smoothness regularization ensures stable affine parameter dynamics under distributional shifts. We validate Meta-LSTM-Affine on FSL and SFDA benchmarks, including Omniglot, MiniImageNet, TieredImageNet, Office-31, MNIST, SVHN, and USPS. Experimental results show that our method consistently outperforms existing approaches such as BN, MetaBN, MetaAFN, and LSTM-Affine, achieving improved stability and adaptation performance with minimal additional computational overhead. Overall, Meta-LSTM-Affine provides a stable and efficient affine modeling mechanism for learning under generalized non-stationary conditions without relying on batch-level statistics. This generalized formulation of non-stationarity allows us to study distributional changes in controlled and widely used benchmark settings, while maintaining relevance to real-world scenarios such as streaming data, continual learning, and time-evolving environments.