Complex Oscillating Language Model (COLM): Coherent Language from Sub-500k Parameters

COLM (Complex Oscillating Language Model) is, to our knowledge, the first complex-valued autoregressive language model. It operates entirely in the complex plane (native torch.cfloat state throughout) using oscillatory neurons as its sole computational primitive, with no self-attention and zero nn.Linear layers in its core blocks. At 498,214 parameters — roughly half the size of the smallest coherent model previously reported in the language modelling literature — it generates coherent philosophical prose on complex theological and technological subject matter, trained in 8.7 hours on a single consumer GPU (RTX 5060 Ti) from a home laboratory with no institutional affiliation, no cloud compute, and no external funding. Adjacent prior work on complex-valued autoregression exists in time-series forecasting (Gu & Jiang 2005; Sykulski et al. 2016), but not in NLP. After AI-assisted and manual literature searches, no prior work using native complex state throughout for generative text modelling has been located. Abstract We present COLM (Complex Oscillating Language Model), a novel autoregressive language model that operates entirely in the complex number plane using oscillatory neurons. COLM replaces the transformer's quadratic-complexity self-attention with an O(N) causal recurrence driven by complex-valued gates, and replaces all learned linear transformations in its core blocks with fixed unitary rotations and element-wise complex oscillatory activations. The architecture contains zero nn.Linear layers in its processing blocks — all transformation is performed by the oscillating activation f(z) = sin(ω ⊙ z + φ) · tanh(z) + b where ω, φ, b ∈ ℂ, routed through fixed energy-preserving complex mixers. At 498,214 parameters — roughly half the size of the smallest coherent model reported by Eldan and Li (2023, TinyStories) — COLM generates thematically coherent philosophical prose on complex theological and technological subject matter. In blind evaluation by GPT-5.4, 84% of COLM's outputs at temperature 1 are estimated as age 13–16 writing, with a mean creativity score of 4.83/10, despite operating under strictly harder conditions than prior work across every axis: a 499-token hybrid tokenizer, conceptually dense subject matter, maximum sampling temperature, and no spell correction. The model achieves a best validation loss of 1.1449 on a 47-million-token corpus derived from 91 public domain works, trains in 8.7 hours on a single consumer GPU (RTX 5060 Ti), and the checkpoint file is 32.2 MB (33,800,192 bytes). COLM builds on the published WiggleGPT result (O'Brien 2025), which demonstrated that a single oscillating neuron can solve XOR — a problem the field assumed required multiple neurons for 56 years following Minsky and Papert (1969). The oscillating activation, promoted from ℝ to ℂ, becomes the sole computational primitive of an architecture class that processes language as interfering complex-valued waves rather than discrete token comparisons. Code: https://github.com/Eden-Eldith/COLM Model: https://huggingface.co/edeneldith/COLM Dataset: https://huggingface.co/datasets/edeneldith/DCDM Predecessor: WiggleGPT (see related identifiers)