Emergence Over Attention: Continuous-Time Phase Synchronization as a Computational Primitive

We present the Waveformer, a 232M-parameter continuous-time language model that replaces the O(N²) softmax attention mechanism with Kuramoto oscillator phase synchronization. Tokens do not attend to each other; they perturb a fixed-size ensemble of coupled oscillators whose synchronization dynamics perform the computation. Position is encoded as an irrational KAM frequency, requiring zero learned parameters. The attention matrix is eliminated via sine-difference mean-field factorization of the Kuramoto coupling term. The macro-architecture is structurally pre-validated using our previously published cross-layer phase coherence analysis (DOI: 10.5281/zenodo.20720827), which identifies universal constructive pockets and conflict anchors across four model scales from 124M to 7B parameters. We pre-train a 207M-parameter variant from scratch for 50,000 steps using a phased developmental curriculum over 184K curated multi-domain entries, achieving perplexity 2.71. Instruction tuning via ChatML masked fine-tuning for 27,000 steps produces a conversational assistant with monotonically decreasing validation loss (2.244 to 2.180) and domain-routed generation behavior. A refined 232M architecture replaces all Q/K/V projections with raw oscillator attention and uses an algebraically reversible Asynchronous Leapfrog integrator for the backward pass, storing only final phase states and reconstructing the trajectory on-the-fly. Forward+backward VRAM is approximately 850MB across sequence lengths 32-512 tokens on an 8GB NVIDIA RTX 3050. A fused CUDA kernel accelerates the mean-field coupling computation.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-18
DOI
https://doi.org/10.5281/zenodo.20741536
Primary Topic
Nonlinear Dynamics and Pattern Formation
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Emergence Over Attention: Continuous-Time Phase Synchronization as a Computational Primitive

Nossair Bajddi
Zenodo (CERN European Organization for Nuclear Research)
Nonlinear Dynamics and Pattern Formation
preprint

Emergence Over Attention: Continuous-Time Phase Synchronization as a Computational Primitive

Nossair Bajddi
preprint en

Abstract

We present the Waveformer, a 232M-parameter continuous-time language model that replaces the O(N²) softmax attention mechanism with Kuramoto oscillator phase synchronization. Tokens do not attend to each other; they perturb a fixed-size ensemble of coupled oscillators whose synchronization dynamics perform the computation. Position is encoded as an irrational KAM frequency, requiring zero learned parameters. The attention matrix is eliminated via sine-difference mean-field factorization of the Kuramoto coupling term. The macro-architecture is structurally pre-validated using our previously published cross-layer phase coherence analysis (DOI: 10.5281/zenodo.20720827), which identifies universal constructive pockets and conflict anchors across four model scales from 124M to 7B parameters. We pre-train a 207M-parameter variant from scratch for 50,000 steps using a phased developmental curriculum over 184K curated multi-domain entries, achieving perplexity 2.71. Instruction tuning via ChatML masked fine-tuning for 27,000 steps produces a conversational assistant with monotonically decreasing validation loss (2.244 to 2.180) and domain-routed generation behavior. A refined 232M architecture replaces all Q/K/V projections with raw oscillator attention and uses an algebraically reversible Asynchronous Leapfrog integrator for the backward pass, storing only final phase states and reconstructing the trajectory on-the-fly. Forward+backward VRAM is approximately 850MB across sequence lengths 32-512 tokens on an 8GB NVIDIA RTX 3050. A fused CUDA kernel accelerates the mean-field coupling computation.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Nonlinear Dynamics and Pattern Formation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.