The BIG SOBOL SEQUENCE: How many simulations do we need for simulation-based inference in cosmology?

How many simulations do we need to train machine learning methods to extract information available from summary statistics of the cosmological density field? Neural methods have shown the potential to extract nonlinear information available from cosmological data. To achieve this requires appropriate network architectures and a sufficient number of simulations for training the networks. This is the first detailed convergence study in which a neural network (NN) is trained to extract maximally informative summary statistics for cosmological inference. We show that currently available simulation suites, such as the Quijote Latin Hypercube with 2000 simulations, do not provide sufficient training data for a generic NN to reach the optimal regime. We present a case study in which we train a moment network to infer cosmological parameters from the nonlinear dark matter power spectrum, where the optimal information content can be computed through asymptotic analysis using the Cramér-Rao information bound. We find an empirical neural scaling law that predicts how much information a NN can extract from highly informative summary statistics as a function of the number of simulations used to train the network, for a wide range of architectures and hyperparameters. Looking beyond two-point statistics, we find a similar scaling law for the training of neural posterior inference using wavelet scattering transform coefficients. To verify our method, we created the largest publicly released suite of cosmological simulations, the B IG S OBOL S EQUENCE (BSQ), consisting of 32 768 Λ cold dark matter N-body simulations uniformly covering the Λ cold dark matter parameter space. Our method enables efficient planning of simulation campaigns for machine learning applications in cosmology, while the BSQ dataset provides an unprecedented resource for studying the convergence behavior of NNs in cosmological parameter inference. Our results suggest that new large simulation suites or new training approaches will be necessary to infer information-optimal parameters from nonlinear simulations.