Grokking Larger DLP with Fixed Model Size: O (1) Representations for Scalable Discrete Logarithm Learning

We study whether the delayed generalization phenomenon (“grokking”) on the finite-field discrete logarithm problem (DLP) can scale to large primes without increasing model size. Our companion paper [Vesterlund, 2026a] demonstrated DLP grokking on small primes (p ≤ 521) but identified a fundamental “tokenizer confound”: both the input embedding and output classifier grow linearly with p, so model parameter count scales as O(p). Here we replace the atomic tokenizer with two O(1) components—Factorized Residue Embedding (FRE) for inputs and Factorized Candidate Output (FCO) for outputs—combined with a CRT-BOTH representation that decomposes the DLP into two coprime subproblems via the Chinese Remainder Theorem. The resulting architecture maintains a fixed ⁓526k parameters regardless of prime size. We demonstrate that this fixed-size model generalizes on DLP across a range of prime sizes (p = 113 to p = 4001), achieving 100% test accuracy at every scale. However, not all configurations exhibit grokking (a memorization plateau followed by delayed generalization): for p = 601 and p = 4001, memorization and generalization times coincide (Tmem = Tgrok), indicating direct generalization rather than grokking. The effective problem size scales with max(a,b) (the larger CRT factor), not with p: the 35× span in p corresponds to a 16× to 256× span in max(a,b). The parameter efficiency advantage grows with p: at p = 4,001 the fixed model is 2.7× smaller than the atomic baseline. Caveat: The CRT decomposition makes per-factor sub-instances few enough that the standard triple-level split allows every sub-instance to appear in both train and test; a sub-instance holdout experiment is needed to determine whether the observed generalization is genuine algorithm learning or sub-instance lookup.