CTP-Hybrid: From Hybrid Architecture to Native Spiking Foundation — A Two-Phase Report on Consumer-GPU Spiking LLMs

We present the complete two-phase development of CTP-Hybrid, a causal-topological pulsing architecture for large language models. Phase 1 established the first reproducible 7B-scale spiking-augmented LLM fine-tuning on an 8 GB consumer GPU (NVIDIA RTX 5060 Laptop), achieving WikiText-2 perplexity of 16.58 with fluent Chinese generation using TD-SCA microcolumns. However, full tri-module joint training (IST+TCP+DSM) failed across seven attempts due to gradient collapse from discrete pulse operations. Phase 2 resolved this fundamentally. We designed three custom autograd.Function implementations—IntervalSpikeEncoder, TemporalCausalMask, and LIFSpikeSurrogate—that provide stable surrogate gradients for previously non-differentiable pulse operations. These functions enabled: (1) the first successful tri-module joint training on a 7B hybrid model with zero NaN, and (2) a fully Transformer-free, 20M-parameter pure spiking baseline trained from scratch on WikiText-2, achieving a perplexity of 13.57. The pure spiking model contains no self-attention, no LayerNorm, and no floating-point FFN, proving that native pulse computation alone can learn natural language. All code, weights, and experiment logs are released.