Mode Effects Between Mobile Web and Telephone Surveys on Patient Experience Scores in South Korea: Secondary Analysis of a Randomized Controlled Trial Under Various Missingness Scenarios

Abstract Background Patient experience surveys are essential tools for assessing health care quality, yet the potential influence of survey mode on patient experience scores remains understudied. This study investigates the mode effects between mobile web and telephone surveys within South Korea’s Patient Experience Assessment. Objective This study aimed to examine the presence and extent of the mode effects of mobile web versus telephone surveys on patient experience scores. The primary outcome was defined as the total score across all 21 survey items, rescaled to 0‐100. Methods This is a secondary analysis using experimental data from a parallel-group randomized controlled trial involving 3200 patients (adults aged ≥19 years, hospitalized >1 day, discharged 2‐56 days before the survey) from 4 general hospitals between October and November 2022, equally allocated to telephone and mobile web survey modes. An independent survey company generated the random allocation sequence using computer-generated random numbers and assigned participants to the survey modes. Due to the nature of the intervention, blinding of participants, interviewers, and outcome assessors was not feasible after assignment. We calculated unadjusted score differences among respondents and estimated adjusted differences accounting for nonresponse using inverse probability weighting (IPW) and multiple imputation (MI) under the missing-at-random assumption. Sensitivity analyses, using the delta-adjustment method based on the missing-not-at-random assumption, assessed robustness to departures from the missing-at-random assumption. Subgroup analyses by sex, age group, and field of care were also conducted. Results Of 3200 patients randomized (1600 per mode), 878 completed the survey (520 mobile web and 358 telephone). Analyses included all randomized participants (n=3200), with nonresponse addressed through IPW and MI. No adverse events were reported in this survey-based study. The total patient experience score was significantly lower in the mobile web group (mean 81.5, SD 16.4) than in the telephone group (mean 84.9, SD 14.3; unadjusted difference –3.41 points, 95% CI –5.51 to –1.31; IPW-adjusted –4.11, 95% CI –6.17 to –2.04; MI-adjusted –4.59, 95% CI –7.45 to –1.73). Similar patterns were observed across most subdomains. Subgroup analyses revealed consistent mode effects across different demographic categories. Sensitivity analyses using the delta-adjustment method confirmed the robustness of these findings under various missing data scenarios. Conclusions Mobile surveys may yield substantially lower patient experience scores than telephone surveys. Unlike previous studies, our study analyzes randomized experimental data under various missingness scenarios and provides evidence that this mode effect is unlikely to be attributable to analytical methods or heterogeneity in respondent characteristics between the 2 survey administration modes. Accordingly, caution is warranted when comparing patient experience scores obtained from traditional telephone surveys with those from mobile surveys. Methodologically, our sensitivity analysis approach provides a robust framework for assessing and addressing potential nonresponse bias in patient experience assessments.