Zero-Exemplar Class-Incremental Learning in Small Language Models via Soft Orthogonal Gradient Projection and Feature Alignment

The deployment of Small Language Models (SLMs, <500M parameters) on edge hardware is constrained by catastrophic forgetting during sequential class-incremental learning (CIL). While storing past text samples (Experience Replay) mitigates classification forgetting, an important limitation is observed: Generative Perplexity Degradation, wherein replaying raw text samples degrades the pre-trained language model backbone (WikiText-2 PPL increases from ~172 to >3,600). To address this trade-off without maintaining episodic text buffers, ANTENA-CIL is introduced: a zero-exemplar structural framework combining three controls: (1) Soft Orthogonal Gradient Projection (Soft-OGP) applying spectral thresholding inspired by random matrix theory to LoRA adapter updates to preserve backbone representations; (2) Head-Level Feature Augmentation (IL2A) generating Gaussian pseudo-features via batched Welford covariance updates to shield classification boundaries from recency bias; and (3) an Entropy-Gated Plasticity Regulator (EGPR) modulating learning rates dynamically based on entropic drift (dH/dt). On DBpedia14 (7-task CIL), ANTENA-CIL achieves 82.88% Average Incremental Accuracy (AIA) with +29.17% Positive Backward Transfer (BWT) while preserving generative perplexity (PPL 172.65) and zero-shot reasoning (PIQA 64.53%)—demonstrating competitive performance relative to O-LoRA (76.50%) on a 50x smaller model (SmolLM-135M). On CLINC-150 (15-task CIL), ANTENA-CIL_SlowDistill achieves 73.69% AIA with competitive zero-shot reasoning (PIQA 59.25% vs 56.91% in Replay) without episodic storage. The framework's gradient projection formalism extends naturally to safety alignment preservation, providing a mathematical basis for deterministic guardrail retention during post-deployment fine-tuning.