Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator

Precision psychiatry aims to identify biomarkers and develop prediction models to improve diagnosis, prognosis, and treatment selection. Despite extensive research, translating findings into clinically useful tools remains challenging. Here we argue that one important contributor to this translational gap is a pervasive misalignment between routine statistical analytic practices and the requirements for real-world decision making. To help address this, we introduce predictive utility analysis: a statistical workflow for estimating the practical significance of biomarkers and prediction models by accounting for outcome base rates and measurement reliability. Starting from effect size or model performance estimates, the workflow simulates expected real-world performance in a target population and summarizes it across complementary metrics including positive and negative predictive value, Precision-Recall AUC, and Net Benefit. Analogous to how power analysis is used for determining the conditions needed to achieve statistical significance, predictive utility analysis can help determine the conditions needed to achieve practical significance. We implement and illustrate this workflow using E2P Simulator, a recently developed open-source web tool that makes predictive utility analysis accessible through interactive simulations and publication-ready visualizations. We demonstrate its application using specific examples in three key areas: diagnostic (depression, Alzheimer’s disease), treatment response (antidepressants), and risk (suicide attempts and psychosis onset) prediction, highlighting conditions under which we may expect clinically meaningful advances. While we focus on translational challenges in psychiatry, the workflow and tools presented here address general statistical challenges and are broadly applicable across biomedical and behavioral sciences.

Authors

Institutions

Publication Details

Journal
eLife
Published
2026-11-19
DOI
https://doi.org/10.7554/elife.110646.2
Primary Topic
Statistical Methods in Epidemiology
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator

Povilas Karvelis, Guillermo Horga, Daniel Felsky, Andreea Oliviana Diaconescu et al.
1 links
eLife
Statistical Methods in Epidemiology
preprint

Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator

Povilas Karvelis, Guillermo Horga, Daniel Felsky, Andreea Oliviana Diaconescu, Anissa Abi-Dargham
preprint en

Abstract

Precision psychiatry aims to identify biomarkers and develop prediction models to improve diagnosis, prognosis, and treatment selection. Despite extensive research, translating findings into clinically useful tools remains challenging. Here we argue that one important contributor to this translational gap is a pervasive misalignment between routine statistical analytic practices and the requirements for real-world decision making. To help address this, we introduce predictive utility analysis: a statistical workflow for estimating the practical significance of biomarkers and prediction models by accounting for outcome base rates and measurement reliability. Starting from effect size or model performance estimates, the workflow simulates expected real-world performance in a target population and summarizes it across complementary metrics including positive and negative predictive value, Precision-Recall AUC, and Net Benefit. Analogous to how power analysis is used for determining the conditions needed to achieve statistical significance, predictive utility analysis can help determine the conditions needed to achieve practical significance. We implement and illustrate this workflow using E2P Simulator, a recently developed open-source web tool that makes predictive utility analysis accessible through interactive simulations and publication-ready visualizations. We demonstrate its application using specific examples in three key areas: diagnostic (depression, Alzheimer’s disease), treatment response (antidepressants), and risk (suicide attempts and psychosis onset) prediction, highlighting conditions under which we may expect clinically meaningful advances. While we focus on translational challenges in psychiatry, the workflow and tools presented here address general statistical challenges and are broadly applicable across biomedical and behavioral sciences.

eLife
Centre for Addiction and Mental Health (CA), University of Toronto (CA), Stony Brook Medicine (US), Stony Brook University (US), Columbia University (US)
Krembil Foundation
Good health and well-being
Statistical Methods in Epidemiology
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon