A continuous honesty index for predicting honest behaviour from demographic factors: a reanalysis controlling for data leakage

We construct a continuous Honesty Index (HI), which combines the frequency and sequentiality of dishonest acts to capture variation that is lost in binary classifications. Demographic and socioeconomic data for 188 participants were paired with behavioural data from a coin-toss prediction task. We tested five algorithms (Linear Regression, Lasso, Elastic Net, Random Forest, XGBoost) in a nested cross-validation scheme with five repetitions of five-fold splits. To address the ceiling concentration (50% of participants achieved the maximum score), we used a logit transformation of the outcome. The original pipeline did SMOTE before cross-validation, leaking synthetic samples into the test folds. Fixing this mistake brought the apparent R-squared from above 0.50 to negative values. Spearman rank correlation between observed and predicted scores was 0.43 (95% CI 0.25, 0.61) and the area under the ROC curve for detecting ceiling-level scores was 0.76 (0.67, 0.84). A 1000-shuffle permutation test ( p = 0.001) showed that both metrics were well above the null. Cross-checked SHAP importance, permutation importance, and bootstrapped Elastic Net coefficients. Only age and good sibling relationship had permutation-importance intervals that did not overlap with zero. Bootstrapped coefficients added single-parent family structure to that set. XGBoost had high SHAP attribution to urban socioeconomic variables, but these variables did not meet any of the three strict criteria, consistent with overfitting to continuous splits rather than a true association. The bottom quartile observed was enriched 3.6-fold in the lowest 10% predicted participants at screening. This property may serve pre-screening in experimental designs where sample selection is expensive.