Calibration-aware gating for budgeted LLM supervision in graph neural network link prediction

Abstract Large-scale graphs underpin critical applications such as financial fraud detection, citation analysis, and biomedical discovery, where link prediction is often performed under severe label scarcity, class imbalance, and operational cost constraints. Recent large language models (LLMs) provide powerful semantic priors that can augment graph learning; however, naïvely querying LLMs is computationally expensive, and direct distillation of their outputs into compact graph models can degrade predictive reliability. In this work, we propose a budgeted LLM-assisted framework for graph link prediction that explicitly accounts for both predictive performance and calibration. The framework consists of four stages: (i) training a backbone graph neural network (GNN) using available labeled edges, (ii) selectively querying an LLM under a fixed query budget, (iii) distilling LLM-provided soft supervision into a lightweight student predictor, and (iv) learning a calibration-aware gating model that adaptively combines base and student predictions on a per-edge basis. The proposed gate relies only on inexpensive uncertainty- and structure-aware signals, such as predictive entropy and node degree, to determine when LLM supervision is likely to be beneficial. We evaluate the proposed approach using both ranking and reliability metrics, including ROC-AUC, Average Precision (AP), Expected Calibration Error (ECE), and Brier score. Experiments on four benchmark datasets—Elliptic, Cora, PubMed, and OGBN-Arxiv—demonstrate that the learned gate consistently improves ranking performance over both the base GNN and the LLM-distilled student under the same query budget, while substantially reducing calibration error. These results highlight that selective, calibration-aware LLM supervision offers a practical and trustworthy pathway for improving graph link prediction in cost-sensitive real-world settings.