Multiple execution of the same MPI application: exploiting parallelism at hotspots with minimal code changes

For a typical climate model, parallelization based on a domain decomposition is a predominant technique to speed up its computation as an MPI (Message Passing Interface) application on an HPC (High Performance Computing) system. In this contribution, it is shown how the potential of simultaneously executing multiple instances of such an MPI application can be exploited to achieve a further speedup with an additional parallelization of compute-intensive loops without data dependencies. Splitting the work at such hotspots between the instances represents an independent level of parallelization on top of the domain decomposition. The climate model can largely be considered as a black box for this second level of parallelization, which makes it very easy to implement. Outside of the hotspots, however, the same computations are performed in all instances to prevent the necessity to synchronize any application data between the instances when a hotspot is reached. Some examples will show that such a conscious acceptance of redundant computations for parallelization approaches is quite common in other disciplines to reduce the time-to-solution. Experimental results show for the example of the additional parallelization of an iceberg and a biogeochemical model, each embedded into an ocean/ice model, how the time-to-solution can be reduced at appropriate efficiency. The implementation of the approach as a supplement to an existing domain decomposition for other simulation models seems promising, if the further reduction of the time-to-solution is in the focus, but a limit for the scalability based on the existing domain decomposition is reached.