STORM on STORM: A Meta-Evaluation of Perspective-Guided Knowledge Curation Through Self-Application

We apply the STORM methodology (Shao et al., 2024) to itself as a subject of inquiry, treating the method's self-application as a stress test. We document eight observations, ranging from the collapse of perspective diversity under nominal role assignment to the structural incompatibility of Wikipedia as a target format for grounded knowledge curation. The central concept that emerges is verification debt: the gap between the cost a system appears to impose and the cost it actually imposes, once the labor of auditing its output is included in the accounting. We propose Verification Debt per thousand words (VD/kw) as a required metric for honest comparison of long-form generation systems.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-25
DOI
https://doi.org/10.5281/zenodo.20846185
Primary Topic
Wikis in Education and Collaboration
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

STORM on STORM: A Meta-Evaluation of Perspective-Guided Knowledge Curation Through Self-Application

Ilya Lankevich
Zenodo (CERN European Organization for Nuclear Research)
Wikis in Education and Collaboration
preprint

STORM on STORM: A Meta-Evaluation of Perspective-Guided Knowledge Curation Through Self-Application

Ilya Lankevich
preprint en

Abstract

We apply the STORM methodology (Shao et al., 2024) to itself as a subject of inquiry, treating the method's self-application as a stress test. We document eight observations, ranging from the collapse of perspective diversity under nominal role assignment to the structural incompatibility of Wikipedia as a target format for grounded knowledge curation. The central concept that emerges is verification debt: the gap between the cost a system appears to impose and the cost it actually imposes, once the labor of auditing its output is included in the accounting. We propose Verification Debt per thousand words (VD/kw) as a required metric for honest comparison of long-form generation systems.

Zenodo (CERN European Organization for Nuclear Research)
Wikis in Education and Collaboration
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.