Correlation-Adjusted Elastic-Net Penalties for Neighborhood Crime Deprivation Modeling in England

A collection of interrelated social variables determines crime deprivation at the neighborhood level in England. In penalized regression, strong intercorrelations pose a fundamental problem for variable selection. Previous research has used Elastic-Net (ENET) or the Least Absolute Shrinkage and Selection Operator (LASSO) to analyze individual waves of the English Indices of Deprivation (IoD) at the Lower Super Output Area (LSOA) level. This leaves questions about what the recently released IoD 2025 reveals about the crime–deprivation relationship, whether variable selection is stable across IoD waves, and whether the Correlation-Adjusted Elastic-Net (CAEN), which embeds actual pairwise Pearson correlations into the penalty matrix, achieves greater sparsity than LASSO and ENET. Using a single consistent Python (3.14.2) implementation that removes cross-software confounds observed in previous CAEN1 assessments, this work applies LASSO, ENET, and CAEN1 to all four publicly available releases of the IoD: 2010 (n = 32,482), 2015 (n = 32,844), 2019 (n = 32,844), and 2025 (n = 33,755). In the CAEN penalty, the diagonal elements guarantee positive semi-definiteness and global convergence of coordinate descent. To ensure an equitable comparison with the classic ENET, the CAEN penalty rescales the penalty factor to correct for double shrinkage. In three out of four waves, CAEN1 outperforms LASSO in terms of sparsity over 50 stratified 70/30 splits using 10-fold cross-validation (CV). Building a six-predictor model that neither LASSO nor ENET could, the CAEN1 set both Income and Employment to zero in 2015. For the mean square error, the differences in prediction accuracy are negligible in magnitude (ΔMSE < 0.002). IDACI, Health and Disability, and Living Environment are the most consistently prominent determinants of neighborhood crime deprivation, according to a 15-year longitudinal study. A corrected resampled t-test confirms that the methods differ little inout-of-sample MSE in three of the four waves, with only a negligible CAEN1 excess reaching significance in 2010. A simulation further shows that in higher-dimensional, strongly collinear, and low signal-to-noise settings, CAEN2 coupling recovers correlated predictor groups that LASSO fragments.