Causal Inference Needs More Than Analysis: The Role of Study Design

In a variety of research fields, causal inference methods have demonstrated potential benefits in identifying and estimating causal relationships from observational data. These methods have started to receive increasing attention in software engineering as well. However, the investigation of causal relationships can be challenging when dealing with observational data. In this paper, we uphold the position that data analysis alone is not enough and a proper design is always needed to detect reliable causal relationships. To this end, we describe the fundamental conditions needed to reveal causal relationships, the challenges of observational studies in satisfying them, and the mechanisms to overcome those challenges.