The Piranha Problem: Large Effects Swimming in a Small Pond

BackgroundIn this work, we discuss an inevitable consequence of having a stable system in which many explanatory variables have large effects: these variables must have large interactions which will be unlikely to cancel either other out to the extent required for general stability or predictability.We call this type of result a "piranha theorem" (Gelman, 2017), the analogy being the folk belief that if one has a large number of piranhas (representing large effects) in a single fish tank, then one will soon be left with far fewer piranhas.If there is some outcome for which studies find large and consistent effects of many different inputs, then we can conclude that some of these effects are smaller than claimed or that multiple explanatory variables are essentially measuring the same phenomenon.Identifying and measuring the effects of explanatory variables are central problems in statistics and drive much of the world's scientific research.Despite the substantial effort spent on these tasks, there has been comparatively little work on addressing a related question: how many explanatory variables can have large effects on an outcome?The present work follows up on Cornfield et al. (1959) and Ding and Vanderweele (2014), considering quantitative constraints in the effects of additional variables.Consider, for example, the problem of explaining voters' behaviors and choices.Researchers have identified and tested the effects of internal factors such as fear, hope, pride, anger, anxiety, depression, and menstrual cycles, as well as external factors such as droughts, shark attacks, and the performance of local college football teams.Many of these findings have been questioned on methodological grounds, but they remain in the public discourse.