Framing Effects on Large Language Models

Large language models (LLMs) have gained popularity for their ability to generate human-like text and automate tasks. With applications in fields such as healthcare and the judicial system, it is essential that LLMs avoid biases. The framing effect occurs when differing presentations of the same information influences decisions, leading to inconsistent responses. Although well-documented in human psychology, its impact on LLMs raises concerns about reliability. To explore this, we tested different LLMs— ChatGPT, Gemini, Perplexity, and DeepSeek—by presenting them with paired yes-or-no questions that were framed both positively and negatively to see whether different framings would affect their answers. We found that ChatGPT was highly susceptible to framing and Gemini was occasionally influenced by it, particularly in healthcare-related queries. Perplexity, while offering the most in-depth explanations, still showed some vulnerability to framing. DeepSeek showed the least susceptibility, changing its answer only once and considering the missing context in its responses. This has significant implications for their use in real-world applications, where framing effects may lead to biased or inconsistent recommendations in areas like healthcare, law, and finance.