Exact-Equivalence Model Transplantation

Exact-Equivalence Model Transplantation Function-Preserving Transfer of Pretrained Transformer Knowledge Into Custom Architectures Author: Phillip HollandProject: DendroMoE BitLayer 1.58 / Exact-Equivalence Model TransplantationDate: 2026Status: Research draft for Zenodo / Hugging Face release Abstract Modern pretrained transformer models contain enormous learned structure, but changing their internal architecture often destroys their pretrained behavior and forces expensive retraining, distillation, or recovery fine-tuning. This paper introduces Exact-Equivalence Model Transplantation, a practical protocol for transferring pretrained transformer knowledge into a custom architecture while preserving the original model’s zero-shot function. The core idea is that a custom target architecture can receive a pretrained model if it exposes an exact-equivalence path for the source model’s computation, preserves the tokenizer-defined coordinate system, maps the embeddings, attention projections, normalization paths, residual flow, feed-forward computation, and output logits, and initializes any new cognitive modules in silent, gated, identity, or bypass-equivalent states. We demonstrate the protocol using DendroMoE-GPT2, a custom architecture in which GPT-2 feed-forward blocks are replaced by a DendroMoE BitLayer containing routed experts, shared neural infrastructure, communication memory, workspace-compatible state, and cognitive metabolism. In CUDA evaluation, the converted DendroMoE-GPT2 model retained GPT-2’s zero-shot perplexity at approximately 100%, with logit cosine similarity approximately 1.0, Q/K/V projection norm max delta 0.0, and near-zero logit error. This shows that a pretrained transformer can be placed inside a new architecture body without losing its original behavior when exact-equivalence constraints are satisfied. The result reframes architecture modification from destructive replacement into knowledge-preserving cognitive expansion: a pretrained model can keep its learned function while new memory, routing, workspace, and state systems are attached for later activation and training. Keywords Exact-equivalence transplantation; model transplantation; transformer architecture; pretrained model transfer; function-preserving transformation; DendroMoE; neural transplant; cognitive expansion; tokenizer coordinate system; shared neural infrastructure; stateful experts; model morphism; GPT-2. 1. Introduction Large pretrained transformers are not only collections of parameters. They are learned computational systems whose embeddings, attention projections, normalization statistics, feed-forward pathways, tokenizer, and output head form a tightly coupled coordinate system. When researchers replace or heavily modify parts of a pretrained transformer, the model often loses coherence because the new architecture no longer matches the functional structure around which the weights were trained. The common response is to retrain, distill, or fine-tune. Distillation transfers knowledge by training a student model to imitate a teacher model, while adapters add small trainable modules to pretrained networks without changing the whole model body. These methods are powerful, but they do not fully answer a deeper architecture question: can a pretrained model be moved into a new computational body instantly, without losing its existing function? Distillation is training-based, and adapters usually preserve the old model by adding side modules rather than replacing the core computation. (arXiv) This paper proposes that pretrained transformer modification should be approached as model transplantation. The goal is not to discard the original model and train a new one. The goal is to preserve the original function exactly while attaching a richer architecture around it. The central thesis is: A pretrained transformer can be transplanted into a custom architecture without zero-shot degradation if the target architecture contains an exact-equivalence path for the source computation and preserves the tokenizer-defined input/output coordinate system. In this work, the target architecture is DendroMoE BitLayer 1.58, a custom stateful expert layer designed to replace feed-forward blocks while exposing additional systems for routing, memory, communication, workspace state, and cognitive metabolism. The key result is that DendroMoE-GPT2 can reproduce GPT-2’s zero-shot perplexity and logits in transplant mode while retaining the ability to later activate the new systems. This is not merely a layer swap. It is a framework for knowledge-preserving cognitive expansion. 2. Relationship to Prior Work This work connects to earlier research on function-preserving transformations, but the emphasis here is different. Net2Net introduced function-preserving transformations that allow a trained neural network to transfer knowledge into a deeper or wider network, accelerating training instead of starting from scratch. Net2Net’s core idea is that one can transform a network while preserving its function. (arXiv) Network Morphism studied how to morph a well-trained neural network into a new architecture while completely preserving the network function. The related line of work provides important precedent for function-preserving architecture change. (Semantic Scholar) This paper’s contribution is not the claim that function preservation has never been studied. The contribution is a practical transformer-focused transplant protocol for modern pretrained language models, emphasizing the full transformer interface: tokenizer coordinate preservation, embedding/output head alignment, Q/K/V norm preservation, residual and normalization preservation, FFN-to-custom-layer equivalence, silent activation of new cognitive systems, and zero-shot perplexity retention. DendroMoE also relates to Mixture-of-Experts models. DeepSeekMoE, for example, explores fine-grained expert specialization and shared experts for efficient large language modeling. (arXiv) The difference is that DendroMoE’s transplant mode is not primarily a training-from-scratch MoE design. It is a custom target body capable of receiving a pretrained transformer function exactly before its extra expert systems are activated. The project also relates to low-bit transformer work such as BitNet and BitNet b1.58, which use ternary-style weight representations for efficiency. (arXiv) In the transplant experiments reported here, quantization is bypassed during exact-equivalence mode because immediate ternary compression can disturb exact function preservation. This separates two phases: first preserve the original model, then optionally activate quantized or low-bit pathways after calibration. 3. Core Concept: Exact-Equivalence Model Transplantation Exact-Equivalence Model Transplantation is the process of moving a pretrained model into a custom target architecture while preserving the original function up to floating-point error. The source model is the pretrained model.The target model is the custom architecture.The transplant succeeds when the target model produces the same logits, perplexity, and internal projection behavior as the source model before any additional training. The target architecture may contain many additional systems. It may contain experts, routers, workspace memory, communication banks, recurrent state, cognitive metabolism, or shared registries. However, these new systems must begin in a state that does not interfere with the original computation. They may be silent, zero-initialized, identity-initialized, bypassed, or strongly gated. This leads to the central principle: Preserve first. Expand second. Improve third. The old model must survive before the new architecture is allowed to wake up. 4. The Tokenizer as the Model’s Coordinate System A pretrained language model’s tokenizer is not a minor preprocessing detail. It is part of the model’s semantic coordinate system. The embedding matrix maps token IDs into hidden vectors. The output head maps hidden vectors back into token logits. If the tokenizer changes, the same token ID may refer to a different text fragment, or the same text fragment may be split into different token IDs. Therefore, the learned relationship between embeddings, internal representations, and logits becomes misaligned. GPT-2 uses a byte pair encoding style tokenization interface as part of its language modeling system. (OpenAI CDN) For exact transplantation, the tokenizer and vocabulary ordering should be preserved unless a separate token-bridge or vocabulary remapping system is trained. A useful analogy is: The tokenizer is the password to the model’s knowledge. More formally: The tokenizer defines the symbolic address space in which the pretrained weights are meaningful. Changing the tokenizer without retraining is like changing the address labels on every memory location in the model. The weights may still contain information, but the input/output keys no longer open the same doors. This paper therefore treats tokenizer preservation as a core requirement for instant coherent transplant. 5. Conditions for Exact Transplantation A custom target model can receive a pretrained transformer exactly when the following conditions are met. 5.1 Token Coordinate Preservation The target model must preserve the tokenizer, vocabulary ordering, token embedding rows, and output logit coordinate system, or provide an exact mapping between the source and target token spaces. 5.2 Embedding Preservation Token embeddings and positional embeddings must be copied or exactly transformed so that source token IDs produce equivalent initial hidden states. 5.3 Attention Preservation The attention system must preserve the source model’s query, key, value, and output projections, either by direct mapping or exact reparameterization. Q/K/V projection norms provide an important diagnostic because even small projection scale errors can