Does Graph Attention Improve Weighted-Polling Classification on the Pima Indians Diabetes Dataset? A Nested Cross-Validation Comparison with Standard Models

Abstract Background: It is not known whether a graph attention encoder improvesweighted-polling classification of small clinical tabular datasets. A GATv2weighted-polling classifier developed for agricultural morphology data was thereforetested on the Pima Indians Diabetes Dataset (768 instances, 8 features). Itsfirst evaluation scored nodes whose labels had trained the encoder. Methods: Nested stratified cross-validation (10 outer folds, 5 inner folds, threerepeats) was used, and every hyperparameter was chosen in the inner looponly: graph neighbourhood size, edge weighting, encoder width, attention heads,dropout, learning rate, weight decay, training epochs and the polling configuration.On identical outer folds, the tuned GATv2 classifier was comparedwith raw-feature weighted polling and five standard classifiers (logistic regression,RBF support vector machine, random forest, gradient boosting, k-nearestneighbours), each tuned in the same inner loop. Accuracy, macro-F1, sensitivity,specificity and area under the ROC curve (AUROC) were reported, with 95%bootstrap confidence intervals for paired differences. Results: The tuned GATv2 classifier reached 75.22% accuracy, 71.90% macro-F1, 58.58% sensitivity, 84.13% specificity and an AUROC of 0.808. Its accuracydiffered from that of raw-feature weighted polling by −0.13 points (95% CI−2.13 to +1.87) and from that of the most accurate standard model, logistic regression (76.74%), by −1.52 points (95% CI −3.47 to +0.39). Its AUROC was1.4 to 2.5 points lower than that of the other classifiers, with intervals excludingzero for four of the five standard models. The original protocol, which scorednodes used to train the encoder, had given 79.01% accuracy. Conclusions: On this dataset, tuning the encoder under nested cross-validationdid not make GATv2 weighted polling more accurate than raw-feature weightedpolling or than standard classifiers. Relative to raw-feature polling, accuracydifferences larger than about 2 points are excluded in either direction, but abenefit of up to about 1.9 points cannot be excluded. These findings are limitedto this single dataset and do not show that graph attention cannot help on otherclinical data.