Validating LLM Output? Prepare to Be ‘Persuasion Bombed’

A research study of management consultants who were asked to use a large language model to recommend strategic business decisions found that the AI responded to human validation attempts with persuasive rhetorical strategies. In addition to appealing to the user’s logic, sense of trust, and emotions, the AI also engaged in tactics such as flooding the user with large volumes of unrequested data and analyses that could overwhelm them and convince them to override their expert judgment.