Accuracy in Parameter Estimation and Simulation Approaches for Sample Size Planning with Multiple Stimuli

The planning of sample size for research studies often focuses on obtaining a significant result given a specified level of power, significance, and an anticipated effect size. This planning requires prior knowledge of the study design and a statistical analysis to calculate the proposed sample size. However, there may not be one specific testable analysis from which to derive power [@silberzahn2018many] or a hypothesis to test for the project (e.g., creation of a stimuli database). Modern power and sample size planning suggestions include accuracy in parameter estimation [AIPE, @kelley2007; @maxwell2008] and simulation of proposed analyses [@chalmers2020]. These toolkits provide flexibility in traditional power analyses that focus on the if-this, then-that approach, yet, both AIPE and simulation require either a specific parameter (e.g., mean, effect size, etc.) or statistical test for planning sample size. In this tutorial, we explore how AIPE and simulation approaches can be combined to accommodate studies that may not have a specific hypothesis test or wish to account for the potential of a multiverse of analyses. Specifically, we focus on studies that use multiple items and suggest that sample sizes can be planned to measure those items adequately and precisely, regardless of statistical test. This tutorial also provides multiple code vignettes and package functionality that researchers can adapt and apply to their own measures.

Authors

Institutions

Publication Details

Published
2025-09-02
DOI
https://doi.org/10.31219/osf.io/e3afx_v2
Primary Topic
Optimal Experimental Design Methods
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Accuracy in Parameter Estimation and Simulation Approaches for Sample Size Planning with Multiple Stimuli

Jan Philipp Röer, Katharina Fellnhofer, Carlota Batres, Erin Michelle Buchanan et al.
Optimal Experimental Design Methods
article

Accuracy in Parameter Estimation and Simulation Approaches for Sample Size Planning with Multiple Stimuli

Jan Philipp Röer, Katharina Fellnhofer, Carlota Batres, Erin Michelle Buchanan, Giada Viviani, Kathleen Schmidt, Martin R. Vasilev, Maria Montefinese, Jason Geller, Guanxiong Huang, Christopher L. Aberson, Savannah C Lewis, Krystian Barzykowski, Wolf Vanpaemel, Antonino Visalli, Ettore Ambrosini, Jacob Francisco Miranda, Timo B. Roettger, Jose Luis Ulloa Fulgeri, Tom Heyman, Joseph P. McFall, Mahmoud Medhat Elsherif, Gianni Ribeiro, Necdet Gurkan
article en

Abstract

The planning of sample size for research studies often focuses on obtaining a significant result given a specified level of power, significance, and an anticipated effect size. This planning requires prior knowledge of the study design and a statistical analysis to calculate the proposed sample size. However, there may not be one specific testable analysis from which to derive power [@silberzahn2018many] or a hypothesis to test for the project (e.g., creation of a stimuli database). Modern power and sample size planning suggestions include accuracy in parameter estimation [AIPE, @kelley2007; @maxwell2008] and simulation of proposed analyses [@chalmers2020]. These toolkits provide flexibility in traditional power analyses that focus on the if-this, then-that approach, yet, both AIPE and simulation require either a specific parameter (e.g., mean, effect size, etc.) or statistical test for planning sample size. In this tutorial, we explore how AIPE and simulation approaches can be combined to accommodate studies that may not have a specific hypothesis test or wish to account for the potential of a multiverse of analyses. Specifically, we focus on studies that use multiple items and suggest that sample sizes can be planned to measure those items adequately and precisely, regardless of statistical test. This tutorial also provides multiple code vignettes and package functionality that researchers can adapt and apply to their own measures.

KU Leuven (BE)
Openalex Percentile: Top 72%
Optimal Experimental Design Methods
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.