Zephyr: A High-Performance Framework for Graph Attention Networks on Heterogeneous Data

The Graph Attention Network (GAT), based on the foundational principles of Graph Neural Network (GNN), represents a significant advancement in the accurate representation and in-depth understanding of complex data structures. However, GAT can’t proficiently handle heterogeneous data, which negatively affects model performance. Existing solutions incur substantial computational overhead when addressing these challenges, compromising efficiency and impeding practical deployment. In this paper, we propose Zephyr to enable GAT to efficiently process heterogeneous data with high performance. In Zephyr, we introduce Dynamic Multi-matrix Design and Spatial Multi-activation Design to eliminate the effects of finiteness of matrix representation capability and difference in heterogeneous spaces. Remarkably, Zephyr shows superior capability in processing heterogeneous data while requiring minimal storage space. Through comprehensive simulations and comparisons with existing schemes, Zephyr demonstrates a substantial performance improvement, increasing accuracy by over 8% on heterogeneous datasets and enhancing fitting speed by approximately 20%. Moreover, with heterogeneous data, it maintains memory overhead below 1.05x and curtails time overhead to 70%.