WTMAD-4: a fair weighting scheme for GMTKN55
The GMTKN55 data set is a collection of standard benchmarks used in molecular quantum chemistry that spans small- and large-molecule thermochemistry, reaction barriers, and non-covalent interactions. Herein, we identify a flaw in the weighted mean absolute deviation (WTMAD) definitions commonly used to quantify performance of various electronic-structure methods for the GMTKN55 set, which under-weight some of its component benchmarks by orders of magnitude. A new WTMAD-4 metric is proposed, based on typical errors observed for a set of ten minimally empirical dispersion-corrected density-functional approximations (DFAs), ensuring fair treatment across all benchmarks. The performance of 115 DFAs is then reassessed using WTMAD-4 and we highlight a literature example where a DFA parametrised by minimising WTMAD-2 underperforms for benchmarks marginalised by that metric.
Authors
- Kyle R. Bryenton (ORCID: https://orcid.org/0000-0001-5716-5314)
- Erin R. Johnson (ORCID: https://orcid.org/0000-0002-5651-468X)
Institutions
- Dalhousie University (CA)
- University of Cambridge (GB)
Publication Details
- Journal
- Physical Chemistry Chemical Physics
- Published
- 2025-12-09
- DOI
- https://doi.org/10.1039/d5cp03741g
- Primary Topic
- Machine Learning in Materials Science
- Type
- article
- Field-Weighted Citation Impact
- 0.00