On the Algorithmic Impact of Scientific Computing on Machine Learning

To alleviate the quadratic complexity of matrix computations, we developed the codes GOFMM and Newton-CG, both with lower computational complexities. We developed GOFMM for highly-parallel execution, showing good scalability up to 128 nodes and runtime benefits against ScaLAPACK or STRUMPACK for a wide range of problems in PDEs, kernel matrices, or graph Laplacians. Newton-CG is developed for GPU data-parallelism in deep learning, performing on par or better than the optimizers SGD and Adam.