A Constraints-based Framework for Transformer Positional Encodings

TAFL-FRAME, the Three-Axis, Five-Layer constraint FRAMEwork, isintroduced as a constraint-based approach for interpreting and derivingpositional encodings in Transformer attention. The framework separatesthe specification of an attention-relevant object, its response toposition-index transformations, and the geometric response of itscontent representations. A comparative analysis maps representativeexisting methods into this specification space. To operationalize theframework, admissible classes of token-local positional-injection mapsare defined, together with a procedure that translates object, symmetry, andgeometry choices into constraint equations.Constraint maps characterize compatible subfamilies, and pipelinesorganize their successive refinement. No general solution procedure isprescribed. A fully worked example starts from admissible affineinjection maps and characterizes a discrete orthogonal-power family byimposing norm preservation on the maps and translation invariance on adot-product score computed with a positional-injection map shared bythe query and key streams. An additional smooth-extension hypothesisyields its rotational specialization. The same exampleidentifies a redundant norm-consistency requirement and shows that thespecified scale-equivariance condition on the score forces positionindependence of the resulting maps at integer positions. The framework thereby supports both thecharacterization of admissible encoding families and the diagnosis ofredundant or overly restrictive requirements.