Adaptive online least squares support vector machine with sliding-window updates and drift-aware eviction for non-stationary data streams

Online learning from non-stationary data streams requires classifiers that continuously adapt to distributional changes without full retraining. Least squares support vector machines (LS-SVM) offer a convenient formulation — training reduces to solving a linear system — but batch retraining is prohibitively expensive for high-volume streams. We develop a sliding-window LS-SVM in which the kernel matrix is maintained by rank-1 Sherman–Morrison–Woodbury (SMW) updates and downdates, with explicit treatment of the LS-SVM bias row and numerical stability monitoring. When the window is full and a new sample arrives, one existing sample must be evicted; the choice of eviction strategy directly affects adaptation to concept drift. We conduct the first systematic comparison of six eviction strategies for LS-SVM, establishing an analytical equivalence between the a-magnitude and Karush–Kuhn–Tucker (KKT) violation criteria specific to the LS-SVM equality-constrained formulation. Experiments on nine datasets (four synthetic, two real-world, three image streams) reveal a scale-invariant empirical crossover at a window-to-concept-length ratio W/CL ≈ 1: first-in, first-out (FIFO) eviction dominates when W/CL < 1 (gap up to 24.9 pp), strategies converge near W/CL = 1, and neither dominates at W/CL > 1; this pattern is confirmed independently for W = 100, 200, and 500 on controlled concept-switching streams. An adaptive strategy combining α-magnitude with a FIFO fall-back triggered by the adaptive windowing (ADWIN) drift detector defaults to α-magnitude throughout — ADWIN fires zero times because the model self-corrects within two to three samples after each concept transition, before the Hoeffding bound accumulates sufficient signal. Condition numbers remain below 102 across 108 stability runs with no forced re-initialisations, confirming the numerical robustness of sequential SMW updates. These findings give practitioners a concrete, scale-invariant rule for choosing between recency-based and utility-based eviction in kernel-based streaming classifiers.