WTMAD-4: a fair weighting scheme for GMTKN55

The GMTKN55 data set is a collection of standard benchmarks used in molecular quantum chemistry that spans small- and large-molecule thermochemistry, reaction barriers, and non-covalent interactions. Herein, we identify a flaw in the weighted mean absolute deviation (WTMAD) definitions commonly used to quantify performance of various electronic-structure methods for the GMTKN55 set, which under-weight some of its component benchmarks by orders of magnitude. A new WTMAD-4 metric is proposed, based on typical errors observed for a set of ten minimally empirical dispersion-corrected density-functional approximations (DFAs), ensuring fair treatment across all benchmarks. The performance of 115 DFAs is then reassessed using WTMAD-4 and we highlight a literature example where a DFA parametrised by minimising WTMAD-2 underperforms for benchmarks marginalised by that metric.

Authors

Institutions

Publication Details

Journal
Physical Chemistry Chemical Physics
Published
2025-12-09
DOI
https://doi.org/10.1039/d5cp03741g
Primary Topic
Machine Learning in Materials Science
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

WTMAD-4: a fair weighting scheme for GMTKN55

Kyle R. Bryenton, Erin R. Johnson
Physical Chemistry Chemical Physics
Machine Learning in Materials Science
article

WTMAD-4: a fair weighting scheme for GMTKN55

Kyle R. Bryenton, Erin R. Johnson
article en

Abstract

The GMTKN55 data set is a collection of standard benchmarks used in molecular quantum chemistry that spans small- and large-molecule thermochemistry, reaction barriers, and non-covalent interactions. Herein, we identify a flaw in the weighted mean absolute deviation (WTMAD) definitions commonly used to quantify performance of various electronic-structure methods for the GMTKN55 set, which under-weight some of its component benchmarks by orders of magnitude. A new WTMAD-4 metric is proposed, based on typical errors observed for a set of ten minimally empirical dispersion-corrected density-functional approximations (DFAs), ensuring fair treatment across all benchmarks. The performance of 115 DFAs is then reassessed using WTMAD-4 and we highlight a literature example where a DFA parametrised by minimising WTMAD-2 underperforms for benchmarks marginalised by that metric.

Physical Chemistry Chemical PhysicsVol. 28(2)
Dalhousie University (CA), University of Cambridge (GB)
Openalex Percentile: Top 66%
Machine Learning in Materials Science
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

WTMAD-4: a fair weighting scheme for GMTKN55 — Kyle R. Bryenton, Erin R. Johnson · Physical Chemistry Chemical Physics (2025) | TGRS Research Map | TGRS