$$\nu$$GNN: Non-Uniformly partitioned full-graph GNN training on mixed GPUs

Abstract Graph neural networks (GNNs) can be adapted to GPUs with high computing capability due to massive arithmetic operations. Compared with mini-batch training, full-graph training does not require sampling of the input graph and halo region, avoiding potential accuracy losses. Current deep learning frameworks evenly partition large graphs to scale GNN training to distributed multi-GPU platforms. On the other hand, the rapid revolution of hardware requires technology companies and research institutions to frequently update their equipment to cope with the latest tasks. This results in a large-scale cluster with a mixture of GPUs with various computational capabilities and hardware specifications. However, existing works fail to consider sub-graphs adapted to different GPU generations, leading to inefficient resource utilization and degraded training efficiency. Therefore, we propose $$\nu$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ν</mml:mi> </mml:math> GNN , a Non-Uniformly partitioned full-graph GNN training framework on heterogeneous distributed platforms. $$\nu$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ν</mml:mi> </mml:math> GNN first models the GNN processing ability of hardware based on various theoretical parameters. Then, $$\nu$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ν</mml:mi> </mml:math> GNN automatically obtains a reasonable task partitioning scheme by combining hardware, model, and graph dataset information. Finally, $$\nu$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ν</mml:mi> </mml:math> GNN implements an irregular graph partitioning mechanism that allows GNN training tasks to execute efficiently on distributed heterogeneous systems. The experimental results show that in real-world scenarios with a mixture of GPU generations, $$\nu$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ν</mml:mi> </mml:math> GNN can outperform other static partitioning schemes based on hardware specifications.