Statistical Parametric Simulation Studies Based on Real Data

ABSTRACT Simulation studies are indispensable for evaluating statistical methods and ubiquitous in statistical research. The most common simulation approach is parametric simulation, where the data‐generating mechanism (DGM) corresponds to a parametric model from which observations are drawn. While many simulation studies aim to give practical guidance on method suitability, parametric simulations in particular are often criticized for being unrealistic. To overcome this drawback, it is sensible to employ real data for constructing the parametric DGMs. However, although real‐data‐based parametric DGMs are widely used, the specific ways in which DGM components are inferred vary, and their implications may not be fully understood. Additionally, researchers often rely on a limited selection of real datasets, with the rationale for their selection being unclear. This paper reviews and formally discusses how components of parametric DGMs can be inferred from real data and how dataset selection can be performed more systematically. By doing so, we aim to support researchers in conducting simulation studies with a lower risk of overgeneralization and misinterpretation. We illustrate the construction of parametric DGMs based on a systematically selected set of real datasets using two examples: one on ordinal outcomes in randomized controlled trials and one on differential gene expression analysis.