Deep Survival Analysis with Temporal Trajectory Embedding for Substandard Life Insurance Pricing: A DeepSurv Modeling Study of Type 2 Diabetes Mellitus

Abstract Substandard (impaired) lives have long been excluded from commercial health in- surance because conventional actuarial tools fail to capture the nonlinear progres- sion of chronic diseases and the high-dimensional interactions among comorbidities. Taking type 2 diabetes mellitus (T2DM) as a case study, this paper constructs a substandard-life rate-making framework centered on the DeepSurv deep survival model. Using 1,883,235 T2DM inpatient claim records with mini-batch training on the full sample, we first expose the failure boundaries of traditional methods: the extra mortality (EM) model suffers a rate inversion (118.22% for T2S1 com- plicated cases versus 88.87% for T2S2 target-organ-damage cases) driven by age- structure confounding; the generalized linear model (GLM) two-part approach and the Tweedie compound model, while capable of decomposing frequency and severity, remain constrained by the proportional-hazards and linear-additivity assumptions; after incorporating province and year controls, the age coefficient in the Gamma severity regression is corrected from −0.0002 to +0.0028 (the directional bias is re- paired), yet remains far below the Poisson frequency coefficient of 0.031, indicating limited capacity of the linear-additive structure in capturing age effects. Deep- Surv, coupling a deep neural network with the Cox partial likelihood and trained on 1,840,774 records, attains a concordance index (C-index) of 0.695. It should be emphasized that the Tweedie/GLM area under the curve (AUC) for binary dis- crimination of excess events reaches 0.715, but the two metrics are not directly comparable—AUC measures binary classification power for whether out-of-pocket expenses exceed a 10,000 RMB threshold, whereas the C-index measures concor- dance in survival time ranking. The core value of DeepSurv lies not in a dominant single discrimination metric, but in its ability to capture high-dimensional interac- tions among comorbidities through a nonlinear risk surface and to provide premium attribution via SHAP—capabilities beyond the reach of linear models. Bootstrap resampling (B = 50) confirms a C-index standard error of 0.0031 with a 95% con- fidence interval of [0.684, 0.695], confirming robust discrimination. Methodologically, our contributions are fivefold. First, we incorporate Elastic Net regularization—embedding both L1 (Lasso) and L2 (Ridge) penalties into the DeepSurv loss function—so that the L1 component drives weak-signal features to- ward zero while the L2 component assigns similar weights to collinear comorbidities 1 (e.g., heart failure and ischemic heart disease), achieving the ideal “equal within group, sparse across groups” feature selection. Empirical comparison demonstrates that Elastic Net (α = 0.5) preserves the C-index while effectively shrinking re- dundant features, resolving the dilemma where pure Lasso arbitrarily selects one variable from a collinear group and pure Ridge cannot perform variable selection. Second, we introduce a long short-term memory (LSTM) network for independent rate-prediction benchmarking against time-varying health trajectories; empirically, its temporal R2 of 0.391 outperforms the GLM cross-section (R2 = 0.197, +99%) yet trails the lagged-mean benchmark (R2 = 0.455), confirming the incremental predic- tive value of temporal information and its future potential as a feature-embedding module for DeepSurv. Third, SHAP (SHapley Additive exPlanations) gradient ap- proximation on the full 1.84 million sample identifies age (mean|SHAP| = 0.269), DX_T2S2 (0.129), chronic ischemic heart disease COMB_I25 (0.123), DX_T2S1 (0.107), and heart failure COMB_I50 (0.107) as the top five risk drivers, with cardiovascular comorbidities approaching or exceeding diabetes staging itself in at- tribution weight. Fourth, we provide a complete mapping from survival probabil- ities to actuarial premiums, introducing a sojourn-time factor A(τ) and a health- management factor H(r) directly into the gross premium formula, and demonstrate SHAP-driven premium attribution with a high-risk case (male, age 65, cardiac co- morbidities, gross premium 14,802 RMB). Fifth, Bootstrap robustness checks con- firm that all key metrics exhibit acceptable standard errors, and the findings are reproducible under repeated sampling. Deep survival analysis with Elastic Net reg- ularization markedly improves the fairness, discrimination, and model parsimony of substandard-life pricing while preserving actuarial interpretability.