GPU-Accelerated, Mixed Precision GMRES(m) with Varied Restarts

GMRES(m) is a popular iterative method for solving linear systems with large, sparse, (possibly) nonsymmetric, and invertible matrices. If a matrix is not positive definite, GMRES(m) will exhibit slow or stagnating convergence behavior. Baker et al. have proposed αGMRES(m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">max</inf>,m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</inf>), which modifies the restart parameter m based on angles between consecutive residual vectors across two cycles of GMRES(m). This method results in faster convergence compared to the baseline of full-precision GMRES(m). Lindquist et al. have proposed a strategy for accelerating GMRES(m) via mixed precision arithmetic. The development of this method is motivated by the recent trend in newer GPUs offering better low-precision performance than high-precision. Although their method takes around the same number of iterations for both mixed precision and the baseline of full-precision GMRES(m), their GPU implementation results in a greater amount of low-precision parallelism offered by newer GPUs. Our work combines optimization strategies proposed by Baker et al. and Lindquist et al. We present a mixed-precision implementation of αGMRES(m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">max</inf>, m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</inf>). Our GPU implementation exhibits 47.16% better performance on average compared to full-precision GMRES(m). We also show 38.89% and 7.8% better performance with mixed-precision αGMRES(m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">max</inf>,m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</inf>) over both mixed-precision GMRES(m) and full-precision αGMRES(m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">max</inf>,m<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</inf>), respectively. We are not aware of any other work that explores the effects that mixed-precision arithmetic and varied restarts have on each other.