You Are the Best Reviewer of Your Own Papers: The Isotonic Mechanism

Fixing AI’s “Peer Review Lottery” Getting a paper into a top AI conference can feel like a lottery, with studies showing reviewer scores are often arbitrary. Now, research from Weijie Su introduces a fix set to reform the field. The new "isotonic mechanism" tackles the crisis by asking authors to do the seemingly counter-intuitive: rank their own submissions from best to worst. The method’s effectiveness lies in its game-theoretic proof that honesty is actually the author’s best possible strategy. By harnessing this truthful self-assessment, the mechanism calibrates noisy and random reviewer scores, ensuring genuine scientific merit rises to the top. After successful large-scale experiments at major conferences, this mechanism isn’t just a theory. It’s being officially adopted by the International Conference on Machine Learning (ICML) in 2026, promising a fairer, more reliable future for millions of AI researchers.

Authors

Institutions

Publication Details

Journal
Operations Research
Published
2025-12-03
DOI
https://doi.org/10.1287/opre.2022.0622
Primary Topic
Expert finding and Q&A systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

You Are the Best Reviewer of Your Own Papers: The Isotonic Mechanism

Weijie Su
Operations Research
Expert finding and Q&A systems
article

You Are the Best Reviewer of Your Own Papers: The Isotonic Mechanism

Weijie Su
article en

Abstract

Fixing AI’s “Peer Review Lottery” Getting a paper into a top AI conference can feel like a lottery, with studies showing reviewer scores are often arbitrary. Now, research from Weijie Su introduces a fix set to reform the field. The new "isotonic mechanism" tackles the crisis by asking authors to do the seemingly counter-intuitive: rank their own submissions from best to worst. The method’s effectiveness lies in its game-theoretic proof that honesty is actually the author’s best possible strategy. By harnessing this truthful self-assessment, the mechanism calibrates noisy and random reviewer scores, ensuring genuine scientific merit rises to the top. After successful large-scale experiments at major conferences, this mechanism isn’t just a theory. It’s being officially adopted by the International Conference on Machine Learning (ICML) in 2026, promising a fairer, more reliable future for millions of AI researchers.

Operations ResearchVol. 74(2)
University of Pennsylvania (US)
Openalex Percentile: Top 49%
Expert finding and Q&A systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.