SAMI Chimera: Sub-Neuronal Weight Surgery, Chimeric Base-to-Chat Synthesis, and Tensor Isometry Certification in DeepSeek-MoE-16B

Post-training preference alignment (RLHF, DPO, KTO) enforces refusal behaviors by establishing persistent attractors in intermediate transformer representations. Current refusal removal techniques, notably directional ablation, operate macroscopically by subtracting a 1-dimensional refusal vector from entire layer weight matrices, modifying 100% of targeted dense parameters and inducing severe reasoning degradation (-10.7% on GSM8K). Building upon the representation engineering framework established in SAMI Omni, this technical report presents Sub-Neuronal Mechanistic Weight Editing (SN-MWE) coupled with Constrained Orthogonal Procrustes Manifold Recovery, evaluated across the DeepSeek-MoE-16B architecture (1,782 experts and 5,466 tensors). By decomposing SwiGLU Feed-Forward Networks into discrete intermediate channel triplets, we prove that refusal mechanisms reside in an ultra-sparse subset (<0.14%) of intermediate channels. Specifically, we formulate and empirically validate:1. Sub-Neuronal Refusal Isolation and Excision: Neutralizing exactly 113 refusal channels across 16 critical experts, modifying only 0.13% of parameters while leaving over 99.86% of weights bit-for-bit untouched.2. Foundation Base Channel Transplantation (Chimeric Synthesis): Refusal-steering channels in the aligned Chat checkpoint are surgically replaced with unaligned Foundation Base columns, preserving conversational syntax while maintaining 100% router gating invariance.3. Constrained Orthogonal Procrustes Stabilization & Isometry Certification: Representations are geometrically stabilized via proper orthogonal rotations in SO(d) with det(R) = +1. A full-model audit across all 5,466 tensors certifies zero numerical anomalies (NaN/Inf) and a mean norm isometry ratio of 0.9991.4. Asymmetric Mixed-Precision Quantization (AsymQ): Packaging dense trunks and router gates at Q8_0 while compressing routed experts to Q3_K/Q4_K, achieving a 9.0–9.3 GB production footprint with 0.0% refusal rate and near-zero reasoning degradation (81.9% on GSM8K, Δ = -0.2%), executed statically in under 4 GB RAM/NVMe.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-16
DOI
https://doi.org/10.5281/zenodo.23417844
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

SAMI Chimera: Sub-Neuronal Weight Surgery, Chimeric Base-to-Chat Synthesis, and Tensor Isometry Certification in DeepSeek-MoE-16B

Spiridon Valentin, VALENTIN SPIRIDON
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
article

SAMI Chimera: Sub-Neuronal Weight Surgery, Chimeric Base-to-Chat Synthesis, and Tensor Isometry Certification in DeepSeek-MoE-16B

Spiridon Valentin, VALENTIN SPIRIDON
article en

Abstract

Post-training preference alignment (RLHF, DPO, KTO) enforces refusal behaviors by establishing persistent attractors in intermediate transformer representations. Current refusal removal techniques, notably directional ablation, operate macroscopically by subtracting a 1-dimensional refusal vector from entire layer weight matrices, modifying 100% of targeted dense parameters and inducing severe reasoning degradation (-10.7% on GSM8K). Building upon the representation engineering framework established in SAMI Omni, this technical report presents Sub-Neuronal Mechanistic Weight Editing (SN-MWE) coupled with Constrained Orthogonal Procrustes Manifold Recovery, evaluated across the DeepSeek-MoE-16B architecture (1,782 experts and 5,466 tensors). By decomposing SwiGLU Feed-Forward Networks into discrete intermediate channel triplets, we prove that refusal mechanisms reside in an ultra-sparse subset (<0.14%) of intermediate channels. Specifically, we formulate and empirically validate:1. Sub-Neuronal Refusal Isolation and Excision: Neutralizing exactly 113 refusal channels across 16 critical experts, modifying only 0.13% of parameters while leaving over 99.86% of weights bit-for-bit untouched.2. Foundation Base Channel Transplantation (Chimeric Synthesis): Refusal-steering channels in the aligned Chat checkpoint are surgically replaced with unaligned Foundation Base columns, preserving conversational syntax while maintaining 100% router gating invariance.3. Constrained Orthogonal Procrustes Stabilization & Isometry Certification: Representations are geometrically stabilized via proper orthogonal rotations in SO(d) with det(R) = +1. A full-model audit across all 5,466 tensors certifies zero numerical anomalies (NaN/Inf) and a mean norm isometry ratio of 0.9991.4. Asymmetric Mixed-Precision Quantization (AsymQ): Packaging dense trunks and router gates at Q8_0 while compressing routed experts to Q3_K/Q4_K, achieving a 9.0–9.3 GB production footprint with 0.0% refusal rate and near-zero reasoning degradation (81.9% on GSM8K, Δ = -0.2%), executed statically in under 4 GB RAM/NVMe.

Zenodo (CERN European Organization for Nuclear Research)
Societatea Română de Anestezie şi Terapie Intensivă (RO)
Openalex Percentile: Top 13%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.