FalsifyAgent: trustworthy scientific reasoning through backward abductive reconstruction and adversarial co-evolution

The trustworthiness of reasoning conclusions is a key criterion for determining whether scientific agents can progress from experimental demonstrations to real-world research applications, and it is essential for preventing conclusions with latent defects from entering experimental design and critical decision-making. Verification restricted to positive evidence coverage of an explicit reasoning chain can leave hidden premises and competing hypotheses untested and can be vulnerable to criterion-aware generation. Drawing on the falsificationist position in the philosophy of science, this paper redefines trustworthiness as the ability of a reasoning conclusion to survive continually evolving falsification attacks and proposes FalsifyAgent, a falsification-based verification framework that integrates backward abductive reconstruction with adversarial co-evolution. We develop a trustworthiness measurement system centered on the Adversarially Robust Credibility Score (ARS), which formalizes credibility as the area under the survival curve of a conclusion subjected to progressively stronger falsification attacks, replacing the conventional binary pass/fail judgment. Under uniform positive conditional bounds on attack coverage and correct final adjudication, we derive a decay bound for the expected ARS of a fixed false proposition. High finite-budget ARS indicates stronger resistance to the specified tests. We introduce a backward abductive reconstruction mechanism that starts from a conclusion and constructs an evidence-demand tree containing must-exist and must-not-exist nodes. This mechanism exposes hidden premises and evidential gaps that forward coverage checks cannot reach as dangling requirements, and combines competing-hypothesis generation with explanatory-power evaluation to actively identify evidential underdetermination and perform graded conclusion downgrading. We design a dynamic verification mechanism based on attacker–defender co-evolution and a library of composable attack operators. A conclusion-information constraint in the bidirectional fitness function prevents vacuous defense, while an evidence-bearing attack requirement prevents unsupported challenges. We also propose a weakness-guided coupling strategy that directs attacks toward high-value targets, including gap nodes revealed by the demand tree and competing hypotheses with small explanatory-score margins. Attack outcomes, in turn, trigger incremental reconstruction of the demand tree, forming a closed verification loop of weakness localization, targeted attack, and repair-driven update. Experiments on the FalsifySci benchmark, which contains 400 propositions spanning biomedicine, chemistry, physics, and computer science, reveal substantial performance differences among verification paradigms. With GPT-4o as the backbone, FalsifyAgent achieves an overall ARS-H of 72.7%, outperforming the shared-corpus retrieval fact-checker by 17.9 percentage points and the four-round debate baseline by 10.6 percentage points. On the hidden-premise subset, it attains a hidden-premise discovery rate of 71.3% with a false-positive rate of only 9.4%, while achieving a downgrade-calibration score of 81.0% on the main set. On the trap-proposition subset, the system penetrates the superficial self-consistency created by retracted literature and outdated data, reaching a false-claim rejection rate of 79.2% and reducing the false-acceptance rate by 21.0 percentage points relative to the shared-corpus retrieval fact-checker. Ablation experiments further show that coupling backward reconstruction and evolutionary adversarial testing through weakness guidance produces a superadditive gain of 1.5 percentage points, confirming the functional orthogonality and synergistic amplification of the framework components. FalsifyAgent provides a promising framework for auditing scientific-agent conclusions, supported by conditional theoretical analysis and evaluation on FalsifySci, with potential applications in prepublication review assistance, decision support, and benchmark-contamination screening. Code and implementation details are available at https://huggingface.co/wmsr22/FalsifySci/tree/main .

Authors

Institutions

Publication Details

Journal
Journal of King Saud University - Computer and Information Sciences
Published
2026-11-19
DOI
https://doi.org/10.1007/s44443-026-01440-9
Primary Topic
Logic, Reasoning, and Knowledge
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

FalsifyAgent: trustworthy scientific reasoning through backward abductive reconstruction and adversarial co-evolution

Hui Xiao, Hongwei Wang, Meng Wang
Journal of King Saud University - Computer and Information Sciences
Logic, Reasoning, and Knowledge
article

FalsifyAgent: trustworthy scientific reasoning through backward abductive reconstruction and adversarial co-evolution

Hui Xiao, Hongwei Wang, Meng Wang
article en

Abstract

The trustworthiness of reasoning conclusions is a key criterion for determining whether scientific agents can progress from experimental demonstrations to real-world research applications, and it is essential for preventing conclusions with latent defects from entering experimental design and critical decision-making. Verification restricted to positive evidence coverage of an explicit reasoning chain can leave hidden premises and competing hypotheses untested and can be vulnerable to criterion-aware generation. Drawing on the falsificationist position in the philosophy of science, this paper redefines trustworthiness as the ability of a reasoning conclusion to survive continually evolving falsification attacks and proposes FalsifyAgent, a falsification-based verification framework that integrates backward abductive reconstruction with adversarial co-evolution. We develop a trustworthiness measurement system centered on the Adversarially Robust Credibility Score (ARS), which formalizes credibility as the area under the survival curve of a conclusion subjected to progressively stronger falsification attacks, replacing the conventional binary pass/fail judgment. Under uniform positive conditional bounds on attack coverage and correct final adjudication, we derive a decay bound for the expected ARS of a fixed false proposition. High finite-budget ARS indicates stronger resistance to the specified tests. We introduce a backward abductive reconstruction mechanism that starts from a conclusion and constructs an evidence-demand tree containing must-exist and must-not-exist nodes. This mechanism exposes hidden premises and evidential gaps that forward coverage checks cannot reach as dangling requirements, and combines competing-hypothesis generation with explanatory-power evaluation to actively identify evidential underdetermination and perform graded conclusion downgrading. We design a dynamic verification mechanism based on attacker–defender co-evolution and a library of composable attack operators. A conclusion-information constraint in the bidirectional fitness function prevents vacuous defense, while an evidence-bearing attack requirement prevents unsupported challenges. We also propose a weakness-guided coupling strategy that directs attacks toward high-value targets, including gap nodes revealed by the demand tree and competing hypotheses with small explanatory-score margins. Attack outcomes, in turn, trigger incremental reconstruction of the demand tree, forming a closed verification loop of weakness localization, targeted attack, and repair-driven update. Experiments on the FalsifySci benchmark, which contains 400 propositions spanning biomedicine, chemistry, physics, and computer science, reveal substantial performance differences among verification paradigms. With GPT-4o as the backbone, FalsifyAgent achieves an overall ARS-H of 72.7%, outperforming the shared-corpus retrieval fact-checker by 17.9 percentage points and the four-round debate baseline by 10.6 percentage points. On the hidden-premise subset, it attains a hidden-premise discovery rate of 71.3% with a false-positive rate of only 9.4%, while achieving a downgrade-calibration score of 81.0% on the main set. On the trap-proposition subset, the system penetrates the superficial self-consistency created by retracted literature and outdated data, reaching a false-claim rejection rate of 79.2% and reducing the false-acceptance rate by 21.0 percentage points relative to the shared-corpus retrieval fact-checker. Ablation experiments further show that coupling backward reconstruction and evolutionary adversarial testing through weakness guidance produces a superadditive gain of 1.5 percentage points, confirming the functional orthogonality and synergistic amplification of the framework components. FalsifyAgent provides a promising framework for auditing scientific-agent conclusions, supported by conditional theoretical analysis and evaluation on FalsifySci, with potential applications in prepublication review assistance, decision support, and benchmark-contamination screening. Code and implementation details are available at https://huggingface.co/wmsr22/FalsifySci/tree/main .

Journal of King Saud University - Computer and Information SciencesVol. 38(10)
National Science Library (CN), Tongling University (CN), Northeast Forestry University (CN)
Logic, Reasoning, and Knowledge
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon