Logistic principal component analysis for combining binary predictors: a comparative study based on area under the receiver operating characteristic curve performance

Combining binary predictors into a single diagnostic score is a common challenge in clinical research. Traditional methods such as logistic regression and unweighted sum scores (USS) are widely used but can be unstable or fail to account for correlations in high-dimensional settings. Logistic Principal Component Analysis (Logistic PCA) is a dimensionality reduction technique for binary data; however, its comparative performance against conventional benchmarks has not been systematically evaluated. We conducted a Monte Carlo simulation study to compare the classification performance of Logistic PCA, Convex Logistic PCA, USS, unpenalized logistic regression, and lasso-penalized logistic regression under varying sample sizes ( \(n = 100\) , 200, 400), predictor correlations ( \(\rho = 0.1\) , 0.5, 0.9), and dimensionalities ( \(p = 10\) , 50, 100; 27 scenarios in total). Each scenario was replicated 1000 times, and the area under the receiver operating characteristic curve (AUC) with 95% confidence intervals was calculated, together with accuracy, sensitivity, specificity, positive and negative predictive values at each method’s Youden-optimal cut-point, mean per-replicate computation time, the Brier score and calibration slope of each method’s predicted probabilities, and the bias and mean squared error of each method’s estimated AUC relative to a population-level reference AUC. Logistic regression failures due to non-convergence or complete separation were also recorded. The methods were further applied to a real-world Chronic Kidney Disease (CKD) dataset comprising 10 binary clinical and laboratory predictors and 233 complete cases. In simulations, Logistic PCA, Convex Logistic PCA, and USS showed stable AUCs across all settings and sample sizes. At \(\rho =0.5\) and \(\rho =0.9\) , Convex Logistic PCA’s AUC was statistically indistinguishable from standard Logistic PCA’s (overlapping 95% CIs, differences \(\le 0.002\) ); at \(\rho =0.1\) , however, Convex Logistic PCA achieved a small but statistically detectable AUC advantage over standard Logistic PCA (non-overlapping CIs, differences of 0.011-0.033 across all nine n -by- p combinations), indicating that the convex reformulation’s benefit, where present, was confined to the low-correlation regime. Unpenalized logistic regression achieved higher AUCs in low-dimensional settings but exhibited severe and non-monotonic instability due to complete separation at higher p , failing to converge in anywhere from a few percent up to 100% of simulation replicates depending on the specific combination of n , p , and \(\rho\) . Lasso-penalized logistic regression converged in all replicates across all 27 scenarios and consistently achieved higher AUC than the three PCA/USS-based methods. Classification metrics beyond AUC preserved the same method ranking as AUC in scenarios where all methods converged reliably. Bias/MSE analysis against a population-level reference AUC confirmed that USS, Logistic PCA, and Convex Logistic PCA were essentially unbiased (mean absolute bias \(\le 0.002\) ), lasso showed a small positive bias (mean bias 0.044), and unpenalized logistic regression showed the largest bias, most severe in scenarios with unreliable convergence (mean bias 0.235). At \(p=100\) , Logistic PCA was consistently one to two orders of magnitude slower per replicate than unpenalized logistic regression, and in most (but not all) n -by- \(\rho\) combinations at \(p=100\) it was also slower than lasso, by up to roughly 24-fold; in one combination ( \(n=400\) , \(\rho =0.9\) ) lasso’s cross-validation overhead made it the slower of the two. Convex Logistic PCA was consistently faster than standard Logistic PCA but remained slower than lasso in most scenarios. Following a reviewer’s request for calibration, we obtained predicted probabilities for USS, Logistic PCA, and Convex Logistic PCA via a secondary logistic recalibration of the outcome on each score, and computed the Brier score and calibration slope for all five methods; calibration slopes were close to 1 for USS, Logistic PCA, Convex Logistic PCA, and unpenalized logistic regression in nearly all scenarios, an expected property of probabilities fitted by unpenalized maximum likelihood on the same data used for evaluation, whereas lasso showed a systematically higher slope (median 1.70, range 1.10–3.40 across all 27 scenarios), indicating shrinkage-induced under-confidence; Brier scores mirrored the AUC-based ranking, with unpenalized logistic regression lowest (best) among converged replicates and Logistic PCA, Convex Logistic PCA, and USS highest. In the CKD dataset, all five methods – including Convex Logistic PCA, added following reviewer feedback – produced identical AUCs of 0.985, although unpenalized logistic regression triggered warnings of complete separation. Logistic PCA and lasso-penalized logistic regression both provide stable alternatives to unpenalized logistic regression in high-dimensional, correlated binary-predictor settings where the latter becomes unstable due to separation. Because Logistic PCA is an unsupervised technique while (lasso) logistic regression is supervised, their AUCs are not directly comparable on equal terms; in our simulations, lasso achieved higher discrimination than Logistic PCA in every scenario while also converging reliably, at moderately higher computational cost than unpenalized regression but well below that of Logistic PCA at high dimensionality. Convex Logistic PCA, evaluated at a reviewer’s request, offered a small but statistically detectable discrimination advantage over standard Logistic PCA at low predictor correlation ( \(\rho =0.1\) ), was indistinguishable from it at moderate-to-high correlation, and was faster in most scenarios. Logistic PCA therefore offers a stability-oriented, outcome-agnostic, and essentially unbiased option for combining binary predictors into a single interpretable diagnostic score, rather than a uniformly higher-discriminating alternative to regularized supervised regression.