A Learned Inverse Is Not a Lifted Curse: What Training Transformers on a Family Knowledge Graph Shows About the Reversal Curse, Why Its Bilinear Geometry Answers a Relation-Level Question the Original Curse Did Not Ask, and the Controls That Would Ca
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research A 2025 study trained decoder-only transformers from scratch on a synthetic family knowledge graph and reported that, under strong weight decay, some models infer withheld parent relations from the child relations they did see, that a bilinear probe in the style of the RESCAL knowledge-graph model reads their middle layers at above 95 percent, and that the same models propagate a single fine-tuned edit to its logical consequences while models without the structure do not. Its authors conclude that the reversal curse is an artifact of how knowledge is encoded rather than an inherent limitation. In this analysis the author reviews and synthesises published results to show that the experiment is real and that it answers a different question from the one the reversal curse posed. The curse, as first reported, concerns a single fact whose reverse appears nowhere and one of whose terms is not an entity the model represents in any other role; the anchor's test asks for an inverse relation between two entities that each appear in four to eight training facts, each repeated about 20,000 times, after the same inverse pair has been demonstrated on 5,000 other entities. Knowledge-graph completion learned years ago that this is the easiest pattern in relational data: a rule that only reads inverse pairs reached 96.4 percent filtered Hits@1 on WN18. What remains genuinely contested is why next-token transformers learn that relation-level rule in some from-scratch studies and fail almost completely in others with the same design logic, and the published differences include the regularisation strength the anchor varies. The paper derives that a bilinear score yields the inverse by transposition only when subject and object are read through the same encoding, a property the anchor's probe assumes rather than tests. It argues that the probe-to-editing correlation (R squared 0.939) does not separate geometry from the generalising solution that produces both. It consolidates 25 published results in a table and a figure, states a procedure for reading reversal and propagation claims, and names six measurements that would settle the open part.
Authors
- Pranay M. Mahendrakar (ORCID: https://orcid.org/0009-0003-7224-029X)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24372086
- Primary Topic
- Topic Modeling
- Type
- preprint