Machine Learning Approach to Estimating Wall-Clock Times and Assigning Resources for a Wide Variety of Quantum Chemical Calculations

Abstract This work presents an artificial neural network (ANN) model, Jaguar Timer, which estimates the wall-clock time of quantum chemical calculations─specifically density functional theory (DFT) and time-dependent DFT (TDDFT) calculations. The ANN model is trained and tested on data generated with the quantum chemical package Jaguar, but the same concept should be applicable to data that originate from any quantum chemical engine. We show that the model accurately predicts DFT and TDDFT wall-clock times with over 97% of test cases having an error within a factor of 4 of the observed runtime, and 77% having an error within a factor of 2. Jaguar Timer is designed to support a diverse collection of Jaguar job types and is applicable to a very wide range of wall-clock durations, from seconds to days. The model supports all atom types, basis sets, and the vast majority of exchange-correlation functionals available in Jaguar. Additionally, a wide selection of other input settings is supported, including pseudospectral (i.e., seminumerical) and analytic integrals, spin-restricted and spin-unrestricted determinants, vacuum and implicit solvent simulations, single-point energies and geometry optimizations, vibrational frequency calculations, and TDDFT single-point energies and geometry optimizations. Jaguar Timer is also capable of estimating wall-clock times of jobs executed in batched mode and jobs that employ multiple CPU cores. The wide scope of the model is its core benefit and significantly broadens the supported job and input types that have been studied in similar works. The model’s generality enables users of Jaguar to predict the length of a given set of jobs within a vast collection of common and advanced application domains. Such predictions empower users to carefully optimize their Jaguar jobs and workflows to reduce overall runtimes. In particular, we show that the Jaguar Timer estimates can be incorporated into a greedy computational resource assignment algorithm to reduce runtimes automatically within a workflow or within job batching.