The Good, the Bad, and the Ugly: The Role of AI Quality Disclosure in Deception Detection

We investigate how low-quality AI advisors, lacking quality disclosures, can help spread text-based deception while seeming to help people detect deception. Participants in our experiment discern truth from deceitful material by evaluating transcripts from a game show that mimicked deceptive social media exchanges on topics with objective truths. We find that when relying on low-quality advisors without disclosures, participants’ truth-detection rates fall below their own abilities, which recovered once the AI’s true effectiveness was revealed. Conversely, high-quality advisor enhances deception detection, regardless of disclosure. We discover that participants’ expectations about AI capabilities contribute to their undue reliance on opaque, low-quality advisors. • Opaque low-quality AI reduces text-based truth detection below the user baseline. • Without disclosure users overestimate poor AI, causing deception to escape detection. • Transparency is irrelevant for good quality AI, vital for poor quality AI. • Results stress policy need for AI quality disclosure.