More information, less precision: Meta‐analysis through random effects

Abstract When several studies or other inputs estimate the same quantity, additional information need not increase precision once the inputs reveal heterogeneity that was not assumed in advance. We study this point in a Gaussian random‐effects model estimated by maximum likelihood. The framework separates within‐input noise from across‐input heterogeneity and, as a consequence, separates two uncertainty measures: the precision with which the common location is estimated and the variability of a new input. We show explicitly how conflicting inputs affect both measures, first in a transparent two‐input example and then with many inputs; we also consider relative input precisions and dependence across inputs. Prediction intervals themselves are not new, but their distinction from confidence intervals is central to the interpretation of the model. Applications to heterogeneous clinical trials and to a controlled experiment in finance show that ignoring either reported standard errors or dependence can materially change the assessment of uncertainty.