The Reverse Turing Test in the Double Chinese Room: Observing GenAI Use in Practical Settings

This paper investigates the impact of Generative Artificial Intelligence (GenAI) on the users in practical contexts, based on empirical experiments. At the core of our analysis are two observations: first, we introduce the “Double Chinese Room” phenomenon by extending John Searle’s classic argument to a scenario in which both the AI and its human user lack a deep understanding of the content they are processing (or are prevented from using their expertise) and are symbolically locked in separate “Chinese Rooms.” Second, and based on the first observation, we propose a reimagined application of the Turing Test to evaluate human capabilities rather than the capabilities of AI systems. Our analysis highlights the complex challenges of understanding and decision-making in GenAI-mediated interactions. We suggest that the ongoing re-evaluation of human performance through the lens of GenAI capabilities represents a pivotal shift that requires a redefinition of the role of humans in automation.