Covariate selection in propensity score matching: A case study of how the Shinkansen has impacted population changes in Japan

• Researchers must pay attention to the selection of covariates when using propensity score matching. • Combining MLE with the AIC and BIC results in methods for screening suitable covariates for propensity score matching. • The approach's effectiveness is shown by comparing covariate models with double/debiased machine learning results. • High-speed railways have significant causal effects on population changes. This study presents a novel covariate selection method that combines Maximum Likelihood Estimation (MLE) with the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), focusing on achieving a balance between model fit and complexity. Our findings emphasize the significant impact of covariates on estimated results in Propensity Score Matching (PSM) analyses. Through case studies, we validate the effectiveness of our proposed method across various PSM approaches, including one-to-one matching, K-nearest neighbors matching, radius matching, kernel matching, and Inverse Probability Weighting (IPW). For researchers constrained to cross-sectional data, our comparisons among different PSM methodologies provide valuable insights. Additionally, we explore the applicability of our method to PSM extensions such as Covariate Balancing Propensity Score (CBPS) and PSM-Difference-in-Differences (DID). Our case study reveals significant causal effects of Japan’s Shinkansen on population changes, with notable growth observed in both cross-sectional and panel data analyses.These findings hold important implications for transportation policy, and we offer recommendations for relevant policymakers based on our results.