Large language model scaling laws for neural quantum states in quantum chemistry

Abstract Scaling laws have been used to describe how large language model (LLM) performance scales with model size, training data size, or amount of computational resources. Motivated by the fact that the field of neural quantum states (NQS) has increasingly adopted LLM-based components, we seek to understand NQS scaling laws, thereby shedding light on the scalability and optimal performance-resource trade-offs of NQS ansätze. In particular, we identify scaling laws that predict the performance, as measured by absolute error and V -score, for transformer-based NQS as a function of problem size in second-quantized quantum chemistry applications. By performing analogous compute-constrained optimization of the obtained parametric curves using a simplified compute estimate model, we find that the relationship between model size and training time is highly dependent on loss metric and ansatz, and does not follow the approximately linear compute-optimal tradeoff found by the Chinchilla laws.