Transparent AI Without Deception, Limitation, and Guidance

Requiring transparency from experts can backfire, resulting in experts inventing misleading or false justifications for the public (deception) or changing decisions to suboptimal ones that are more easily defended (limitation or guidance). Onora O'Neill and C. Thi Nguyen claim this dynamic occurs with human experts, and multiple authors have applied the same argument to transparent AI. However, little analysis has been done to evaluate whether the argument applies to transparent AI. In this paper we remedy this by examining the extent to which their argument applies to contemporary approaches to transparent AI. We show that many contemporary approaches to enhancing AI transparency, especially post-hoc interpretability methods, are not vulnerable to this dynamic: this is because they do not incentivize changes in AI outputs or alter the deployed version of a model. Nevertheless we show that these problems could in theory or in practice arise in some AI systems, for example in LLMs trained with Reinforcement Learning from Human Feedback over reasoning traces. Deception, limitation, and guidance are therefore not ubiquitous problems in many contemporary approaches to AI transparency, but do pose challenges for certain system designs.