From points to clusters: rethinking shadow dataset generation for enhanced membership inference

Machine learning models are extensively deployed in diverse applications, and some are trained using sensitive data such as medical and business datasets, which poses significant privacy risks. A Membership Inference Attack (MIA) aims to determine whether a data record was used in model training. Many MIA techniques use shadow model-based attacks (such as LiRA), which depend critically on the quality of the shadow dataset, yet existing work constructs shadow datasets by random sampling from a population distribution, ignoring the subpopulation structure of the feature space. In this study, we propose a clustering-based shadow dataset generation framework that partitions the population into semantically coherent subgroups before drawing shadow training sets, ensuring shadow models are exposed to the same local decision-boundary structure as the target model. Our primary method, PQ-K means (PQK), applies product quantisation to L2-normalised VGG16 features before K-means clustering, yielding subpopulation-aware shadow datasets without requiring class labels and remaining efficient on large-scale datasets. Our experiments on CIFAR-10 with a Wide ResNet-28-10 target and 16 shadow models show that PQK outperforms the LiRA baseline at the strictest operationally relevant threshold (True Positive Rate (TPR) at 0.1% False Positive Rate (FPR)): TPR of 0.0493 (PQK) versus 0.0453 (LiRA online), and 0.0454 (PQK) versus 0.0424 (LiRA offline). An ablation study pinpoints the cause: standard K-means (TPR 0.0402) and GMM with PCA-128 (TPR 0.0436) both underperform PQK, and cluster quality analysis (NMI, Purity, ARI) confirms that PQK forms the most semantically coherent clusters among the unsupervised methods. Bootstrap confidence intervals and label-permutation significance tests confirm that PQK’s advantage at [email protected]%FPR is statistically significant ( \(p < 0.002\) ). These findings establish that semantically structured shadow datasets improve high-confidence membership inference.