Deterministic Layer Freezing in Autoregressive Language Models via Continuous Phase Coherence Analysis
We introduce a method for identifying which layers in pre-trained autoregressive language models can be frozen, which should be retrained, and which actively interfere with model performance without running a single gradient step. By mapping per-layer activations to complex oscillator phase perturbations in a Kuramoto dynamical system, we compute a cross-layer interference matrix that reveals the global structural relationships between all layers simultaneously. Applied across four models spanning 124M to 7B parameters (GPT-2 Small, Qwen2.5-0.5B, Qwen2.5-1.5B, Mistral-7B), we discover three universal structural properties: (1) early layers 0-4 form a universal constructive foundation across all domains, (2) at 28+ layers, a second constructive pocket emerges in middle layers, and (3) a conflict anchor layer at approximately 60-70% depth maximally interferes with other layers in every model tested. Phase-guided fine-tuning on GPT-2 Small freezing layers 0-3, pruning the conflict anchor layer 6, and retraining only 30% of parameters improved perplexity by 41.7% on multi-domain text while reducing trainable parameters by 70%. The method requires no training to produce layer-level guidance and generalizes across model architectures and scales.
Authors
- Nossair Bajddi
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-16
- DOI
- https://doi.org/10.5281/zenodo.20720827
- Primary Topic
- Topic Modeling
- Type
- preprint