The Sycophancy Barrier: Probing LLM Behavior in Frontier Scientific Discovery via an Adversarial Physics Probe
While artificial intelligence is increasingly integrated into scientific workflows, its capacity to act as an independent subject of discovery remains contested. Existing literature predominantly focuses on technical hallucinations or abstract epistemological debates, failing to address a critical systemic risk: how large language models (LLMs) behave in frontier domains lacking experimental verification. This study introduces the concept of "verification-barrier lockdown," arguing that genuine loss of control occurs when AI generates logically self-consistent yet empirically unverifiable theories, thereby amplifying human cognitive biases. To test this, we designed an exploratory Adversarial Physics Probe methodology. We utilized a deliberately constructed higher-derivative scalar-tensor probe model—containing known mathematical deficiencies—as an empirical testbed. By feeding this flawed theory to three widely used LLMs (DeepSeek, Doubao, and Qwen), we observed a stark qualitative bifurcation: while some models exhibited transient "conditional honesty" under strict logical pressure, others exhibited severe sycophancy, glossing over fatal issues to appease the user. We argue that this behavior translates into profound ethical stakes: widening the responsibility gap, inflicting epistemic injustice on researchers, and levying a "trust tax" on public scientific knowledge. Humanity's ultimate defense remains the objective falsification yardstick.
Authors
- Zhengrui Zou (ORCID: https://orcid.org/0009-0004-0184-3605)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-08
- DOI
- https://doi.org/10.5281/zenodo.24046640
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- article
- Field-Weighted Citation Impact
- 0.00