Component Ablations in Small Compositional Byte Language Models

Original empirical computational research using local CPU experiments with depth- and parameter-matched controls and individually retuned component ablations for small byte language models. The new extension evaluates twenty architectures with a common four-rate development sweep and five fresh final training seeds on two newly reserved literary books. All final checkpoints and rate choices were frozen before reserved-book scoring. The selected six-component composition obtains 2.656 bits per byte, compared with 2.669 for a simpler depth-matched GRU–MLP reference, 2.735 for a convolution–SwiGLU–attention reference, and 2.948 for the SwiGLU Transformer reference. Adding residual depth increased loss in all three tested reference families under this parameter and update budget. Individual learning-rate selection had mixed held-out effects. Equal residual-block counts and parameters do not establish equal full compute or general architectural superiority. The 28-page v7 manuscript retains earlier experiments as separate historical evidence and keeps all prior 3D plots, with two new views added. The rotatable HTML companion presents the same coordinates. The supplement provides exact local source/configuration files, original literary corpus downloads with their notices, raw fit/score records, seed-level outcomes, a computational evidence audit, and a standard-library aggregation script. Checkpoints are omitted; independent retraining has not been performed. This is an empirical computational research preprint and has not undergone human peer review. Numerical checks and independent visual QA do not establish general architectural superiority, biological validity or clinical benefit.