Extending MPI Correctness Benchmarking to the Fortran Language

MPI correctness benchmarks evaluate the implementation quality of MPI correctness tools on a standardized set of tests. However, existing correctness benchmarks are limited to C, neglecting support for Fortran, the only other language that the MPI standard supports. Consequently, past evaluations of correctness tools focused solely on C, although some support error-checking of Fortran MPI codes. To alleviate this, we port the test generation logic of the most recently introduced MPI correctness benchmark, MPI-BugBench, to Fortran. We explore language-specific porting challenges and perform a comparative accuracy evaluation of the dynamic MPI correctness tool MUST on both C and Fortran. Our results show that MUST’s accuracy is largely consistent across languages, with a notable exception in type checking due to a required software dependency not supporting Fortran. Additionally, we uncovered bugs in both the OpenMPI Fortran bindings, and the MPI-BugBench test case generator.