Model-based algorithms to ascertain smoking in administrative health data: a registry-based validation study

Accurate measurement of smoking in population-based administrative health data (AHD) poses challenges due to the indirect nature of smoking-related information collection. While most studies use rule-based algorithms (RBAs) based on diagnosis codes, model-based algorithms (MBAs) utilizing machine learning (ML) with diverse data features might have better sensitivity and accuracy. We developed ML model-based algorithms (MBAs) for ascertaining smoking in AHD and compared them to RBAs. We conducted a retrospective cohort study using AHD (hospital abstracts, medical claims, and prescription drug records) from April 1, 2012, to March 31, 2020, from Manitoba, Canada. The study included adults (≥ 18 years) from a clinical registry containing self-reported current smoking. Clinical data were linked with up to five years of hospital records, physician billing claims, and prescription medication records. RBAs were based on diagnosis codes for tobacco use and nicotine dependence medication. MBAs, constructed using Random Forest (RF) and Least Absolute Shrinkage and Selection Operator (LASSO) models, included smoking indicators, comorbid condition, and sociodemographic factors. Training and test datasets were used to develop and evaluate the MBAs, respectively. Sensitivity, specificity, positive and negative predictive values (PPV, NPV), balanced accuracy, and their 95% confidence intervals (CIs) were estimated. The cohort comprised 24,718 individuals (88.6% female); prevalence of current smokers was 10.0%. A comprehensive RBA had sensitivity 23.3% (95% CI: 20.3–26.5), specificity 98.9% (95% CI: 98.7–99.2), and PPV 70.9% (95% CI: 65.1–76.1). An MBA based on RF had sensitivity 66.8% (95% CI: 63.3–70.2), specificity 77.8% (95% CI: 76.8–78.8), and PPV 25.1% (95% CI: 23.8–26.4). NPV was consistently above 90.0%. MBAs had higher balanced accuracy than RBAs. Stratified analyses by sex and residence location revealed differences in estimates for MBAs and the RBAs. The number of years of AHD did not affect the MBA results. While MBAs had better sensitivity, RBAs had better specificity. Our study highlights the potential of comprehensive data integration and ML methods to improve the sensitivity and accuracy of smoking identification in AHD. Balancing accurate smoker identification with the risk of false positives is crucial when choosing an algorithm to ascertain current smokers using AHD.