Deterministic Layer Freezing in Autoregressive Language Models via Continuous Phase Coherence Analysis

We introduce a method for identifying which layers in pre-trained autoregressive language models can be frozen, which should be retrained, and which actively interfere with model performance without running a single gradient step. By mapping per-layer activations to complex oscillator phase perturbations in a Kuramoto dynamical system, we compute a cross-layer interference matrix that reveals the global structural relationships between all layers simultaneously. Applied across four models spanning 124M to 7B parameters (GPT-2 Small, Qwen2.5-0.5B, Qwen2.5-1.5B, Mistral-7B), we discover three universal structural properties: (1) early layers 0-4 form a universal constructive foundation across all domains, (2) at 28+ layers, a second constructive pocket emerges in middle layers, and (3) a conflict anchor layer at approximately 60-70% depth maximally interferes with other layers in every model tested. Phase-guided fine-tuning on GPT-2 Small freezing layers 0-3, pruning the conflict anchor layer 6, and retraining only 30% of parameters improved perplexity by 41.7% on multi-domain text while reducing trainable parameters by 70%. The method requires no training to produce layer-level guidance and generalizes across model architectures and scales.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-16
DOI
https://doi.org/10.5281/zenodo.20720827
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Deterministic Layer Freezing in Autoregressive Language Models via Continuous Phase Coherence Analysis

Nossair Bajddi
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Deterministic Layer Freezing in Autoregressive Language Models via Continuous Phase Coherence Analysis

Nossair Bajddi
preprint en

Abstract

We introduce a method for identifying which layers in pre-trained autoregressive language models can be frozen, which should be retrained, and which actively interfere with model performance without running a single gradient step. By mapping per-layer activations to complex oscillator phase perturbations in a Kuramoto dynamical system, we compute a cross-layer interference matrix that reveals the global structural relationships between all layers simultaneously. Applied across four models spanning 124M to 7B parameters (GPT-2 Small, Qwen2.5-0.5B, Qwen2.5-1.5B, Mistral-7B), we discover three universal structural properties: (1) early layers 0-4 form a universal constructive foundation across all domains, (2) at 28+ layers, a second constructive pocket emerges in middle layers, and (3) a conflict anchor layer at approximately 60-70% depth maximally interferes with other layers in every model tested. Phase-guided fine-tuning on GPT-2 Small freezing layers 0-3, pruning the conflict anchor layer 6, and retraining only 30% of parameters improved perplexity by 41.7% on multi-domain text while reducing trainable parameters by 70%. The method requires no training to produce layer-level guidance and generalizes across model architectures and scales.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.