Recombination or Discovery? A Retrieval-Grounded Novelty Audit of Machine-Generated Research Papers

Fully machine-generated research papers now arrive by the hundreds, but whether they discover anything or merely recombine existing ideas remains unmeasured. We present a retrieval-grounded, protocol-frozen novelty audit of 166 machine-generated papers (the FARS corpus) against 166 topic-matched ICLR 2025 submissions. We decompose each paper into contribution claims along four facets (purpose, mechanism, evaluation, domain; 549 machine and 494 human contributions), retrieve prior art per facet under per-paper submission-date cutoffs, and classify each contribution as covered, recombination, or facet-novel using a pre-registered two-judge protocol with tiebreak arbitration and deterministic state derivation. Machine contributions are judged facet-novel more often than human ones (56.1% vs. 35.6%). We argue this comparison must not be taken at face value: a standard-anchored adversarial re-audit flags 15-25% of purpose-novel verdicts as potentially covered, and removing the standard anchor drives the refutation rate to 100% on the same items. LLM novelty verdicts therefore cannot be certified by another LLM; we treat automated novelty rates as exploratory upper bounds (a 106-pair gold prior-art audit finds the deployed retrieval surfaces the known prior art for only 25-29% of pairs per arm) and release a frozen, blinded human-calibration protocol as an artifact for future validation. A companion integrity audit of 306 Agents4Science 2025 submissions finds hard evidence of data fabrication in 0/47 accepted versus 16/197 rejected submissions (one-sided Fisher p=0.029), an association between honest disclosure and acceptance rather than evidence of reviewer detection.