Enhanced parallelisation for R-matrix with time-dependence, double ionisation

To 'view' electrons in an atom or molecule, we must capture their motion on a time-scale comparable to their interactions. Ultrashort laser pulses can image and control electrons in motion. Free-electron laser facilities operating in VUV, XUV and X-ray regimes have opened up new approaches to investigate ultra-fast dynamics on femtosecond and sub-femtosecond timescales. The complex physics induced by high-energy laser pulses requires close interaction with accurate theory to interpret experimental findings. R-matrix with time-dependence (RMT) is the world’s most sophisticated code for correlated laser-driven multielectron dynamics in atoms and molecules driven by arbitrarily polarised pulses, including relativistic effects in atomic systems. A major extension treats double-ionisation phenomena, wherein two interacting electrons may be ejected from an atom simultaneously. This massively increases complexity and computational workload. RMT now divides physical space into three regions. The first surrounds the nucleus, utilizing a basis expansion for fully correlated multi-electron effects. The single-ionisation and the new double-ionisation regions, with respectively one and two electrons far from the nucleus, utilize finite-difference grids. Each region adopts separate numerical schemes requiring distinct multi-layered parallelisation and communication to manage flow of information between regions. Performance issues with the new region restricted efficient operation to near 1500 cores. We describe a complex additional layer of nested MPI in the new region to improve performance and match sophistication in the other regions. The project also investigates systematic choice of distribution of MPI tasks between regions to identify optimal load-balancing and performance strategies, crucial for large-scale ARCHER2 calculations.