A Learned Inverse Is Not a Lifted Curse: What Training Transformers on a Family Knowledge Graph Shows About the Reversal Curse, Why Its Bilinear Geometry Answers a Relation-Level Question the Original Curse Did Not Ask, and the Controls That Would Ca

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research A 2025 study trained decoder-only transformers from scratch on a synthetic family knowledge graph and reported that, under strong weight decay, some models infer withheld parent relations from the child relations they did see, that a bilinear probe in the style of the RESCAL knowledge-graph model reads their middle layers at above 95 percent, and that the same models propagate a single fine-tuned edit to its logical consequences while models without the structure do not. Its authors conclude that the reversal curse is an artifact of how knowledge is encoded rather than an inherent limitation. In this analysis the author reviews and synthesises published results to show that the experiment is real and that it answers a different question from the one the reversal curse posed. The curse, as first reported, concerns a single fact whose reverse appears nowhere and one of whose terms is not an entity the model represents in any other role; the anchor's test asks for an inverse relation between two entities that each appear in four to eight training facts, each repeated about 20,000 times, after the same inverse pair has been demonstrated on 5,000 other entities. Knowledge-graph completion learned years ago that this is the easiest pattern in relational data: a rule that only reads inverse pairs reached 96.4 percent filtered Hits@1 on WN18. What remains genuinely contested is why next-token transformers learn that relation-level rule in some from-scratch studies and fail almost completely in others with the same design logic, and the published differences include the regularisation strength the anchor varies. The paper derives that a bilinear score yields the inverse by transposition only when subject and object are read through the same encoding, a property the anchor's probe assumes rather than tests. It argues that the probe-to-editing correlation (R squared 0.939) does not separate geometry from the generalising solution that produces both. It consolidates 25 published results in a table and a figure, states a procedure for reading reversal and propagation claims, and names six measurements that would settle the open part.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-19
DOI
https://doi.org/10.5281/zenodo.24372086
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A Learned Inverse Is Not a Lifted Curse: What Training Transformers on a Family Knowledge Graph Shows About the Reversal Curse, Why Its Bilinear Geometry Answers a Relation-Level Question the Original Curse Did Not Ask, and the Controls That Would Ca

Pranay M. Mahendrakar
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

A Learned Inverse Is Not a Lifted Curse: What Training Transformers on a Family Knowledge Graph Shows About the Reversal Curse, Why Its Bilinear Geometry Answers a Relation-Level Question the Original Curse Did Not Ask, and the Controls That Would Ca

Pranay M. Mahendrakar
preprint en

Abstract

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research A 2025 study trained decoder-only transformers from scratch on a synthetic family knowledge graph and reported that, under strong weight decay, some models infer withheld parent relations from the child relations they did see, that a bilinear probe in the style of the RESCAL knowledge-graph model reads their middle layers at above 95 percent, and that the same models propagate a single fine-tuned edit to its logical consequences while models without the structure do not. Its authors conclude that the reversal curse is an artifact of how knowledge is encoded rather than an inherent limitation. In this analysis the author reviews and synthesises published results to show that the experiment is real and that it answers a different question from the one the reversal curse posed. The curse, as first reported, concerns a single fact whose reverse appears nowhere and one of whose terms is not an entity the model represents in any other role; the anchor's test asks for an inverse relation between two entities that each appear in four to eight training facts, each repeated about 20,000 times, after the same inverse pair has been demonstrated on 5,000 other entities. Knowledge-graph completion learned years ago that this is the easiest pattern in relational data: a rule that only reads inverse pairs reached 96.4 percent filtered Hits@1 on WN18. What remains genuinely contested is why next-token transformers learn that relation-level rule in some from-scratch studies and fail almost completely in others with the same design logic, and the published differences include the regularisation strength the anchor varies. The paper derives that a bilinear score yields the inverse by transposition only when subject and object are read through the same encoding, a property the anchor's probe assumes rather than tests. It argues that the probe-to-editing correlation (R squared 0.939) does not separate geometry from the generalising solution that produces both. It consolidates 25 published results in a table and a figure, states a procedure for reading reversal and propagation claims, and names six measurements that would settle the open part.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon