Banya AI 3: Orthogonal Data-Operation Tokens and Convolutional Scale-Depth Structure — Label-Free Emergence of Operation-as-Weight

We view language as an orthogonal product of data (content: what a thing resembles) and operation (transformation: how it changes things). In this view, a single token exists as an orthogonal pair of two embeddings, a data plane and an operation plane. We call this orthogonal-pair token a bi-token. Its defining feature is that which plane is data and which is operation is never labeled in advance — only geometry (orthogonality) and positional asymmetry (an additive slot versus a multiplicative slot) are enforced. The data plane is content added to the residual stream, and the operation plane is a multiplicative gate that opens and closes the channel valves of the next token. Without labeling which token is data and which is operation, enforcing only geometry and positional asymmetry makes the operation plane emerge as a consistent operator that acts in a consistent direction on any data. We measure this and show the following. The directional consistency of operators is 10.1 times that of random; 72.5 percent of distinct operations are orthogonal to each other; and turning the operation plane on lowers the cross-entropy of next-word prediction by 2.79 (from 2.99 to 0.20, over 200 windows of the bundled self-made elementary dialogue corpus). Token identities split to the data and operation extremes, and the middle is empty. Channel mixing uses a circulant filter bank instead of a dense matrix, forming a ladder of depth and scale in which shallow layers become broad brushes and deep layers fine pens. This sequential composition carries the syntagmatic relation (order) of language, and order is handled by the mixing heads. All 16 mixing heads look at the same distance at the start of training and, by the end of training, differentiate to divide up different distances from 39 to 66 positions in the past. Finally, we measure the assembly discrimination of stem-ending combinations never seen in training using minimal pairs. On 198 unseen combinations, accuracy is 86.9% under the reversal distractor and 87.7% under the ending-internal swap distractor, far above the 50% chance level, and it holds even while a syllable-bigram baseline collapses to 47.7% on the hard distractor. We make no causal claim that the orthogonal structure is the cause, as no control model is available. This record contains the English edition (_en.pdf) and the Korean original (_kr.pdf). Part 3 of the six-part Banya AI research series. Part 1: https://doi.org/10.5281/zenodo.21385447. Part 2: https://doi.org/10.5281/zenodo.21385516. All measured figures are reproducible with the public code package: https://ubmscoin.github.io/banya/banya_ai_paper_code/ (code Apache-2.0). Frozen model checkpoints: https://doi.org/10.5281/zenodo.21383724