Preference-Graph Repair and Retrieval Effectiveness: An Empirical Decomposition

Abstract Preference-graph repair removes contradictory comparisons, but structural consistency and retrieval effectiveness are different objectives. Their separation is well understood in principle; less clear is where the effect of repair stops between the graph, the extracted ranking and its evaluated relevance profile. We investigate this question through matched unrepaired, greedy and exact-repair comparisons that hold candidates, preference evidence, extraction and evaluation fixed. The central analysis decomposes responses on 213 selected queries across four datasets. Greedy and exact repair change 180 and 160 rankings, respectively, yet measured utility remains unchanged in 79.4% and 84.4% of those changed rankings. Two mechanisms account for these zeros: pools without a judged-positive candidate and positive-containing pools whose evaluated top-ten relevance profile is preserved. The latter occurs in 74 cases per repair branch and across every dataset and provider-availability stratum. Thus missing judged opportunity alone does not explain the observed inactivity. Query-block inference supplies no Holm-adjusted rejection in the 35 exact-versus-greedy comparisons, while a complete small-instance diagnostic shows that equal-cost repairs can yield different utility. These findings distinguish an intervention that changes rankings from one that changes measured relevance. They extend earlier construction-dependent evidence through conditional frequencies and relevance-profile attribution, rather than a new repair algorithm. Nonrandom selection, incomplete judgments, sparse informative support and an unreliable compact-model comparison instrument limit generalization; the results establish neither equivalence nor a universal absence of benefit.