Performance Trade-offs in GNN Inference: An Early Study on Hardware and Sampling Configurations

Inferencing over Graph Neural Network (GNN) has recently gained traction in applications like recommender systems and fraud detection. However, building a high through-put, low latency GNN serving system for large graphs in real-world applications is challenging due to the irregular memory access patterns and high host−device data transfer times. In this poster, we study the effects of various hardware and sampling configurations on the latency and performance of GNN serving, which will help build scalable GNN serving systems. Our results show that sampling on CPU and inference on GPU, popular for training, does not do well for inferencing. Smaller fan-outs give reasonable accuracies but suffer from high variability while GPU-based sampling fails with high fan-outs, making Unified Virtual Addressing (UVA) based sampling a preferred alternative.