Differences in perceived experiences of AI-based versus human psychological support among dual users: paired comparisons, hierarchical regression, and repeated-measures sensitivity analysis

Conversational artificial intelligence (AI) is increasingly used for psychological support, but limited evidence has compared users’ perceived experiences of AI-based support and human counseling within the same individuals. This study examined domain-specific perceived differences between AI-based psychological support and human counseling among adults who had used both modalities, and explored individual and use-related correlates of these experiences. A cross-sectional online survey was conducted among 233 Chinese adults who reported experience with both AI-based psychological support tools and human counseling. Participants completed two parallel 11-item experience scales assessing perceived acceptance, understanding, goal alignment, collaboration, professional competence, response efficiency, safety/privacy, trust, reduced concern about being judged, willingness for deep self-disclosure, and disclosure of negative emotions. Because the response-efficiency item was not strictly measurement-equivalent across modalities, a reduced 10-item composite score excluding this item was analyzed using a paired-samples t test. A repeated-measures linear mixed-effects model was additionally fitted as a sensitivity analysis to account for the within-person paired structure and to examine whether modality differences varied according to individual characteristics. Paired Wilcoxon signed-rank tests with Monte Carlo two-tailed p values and Holm correction compared AI and human counseling ratings. Hierarchical regression models examined correlates of AI experience, human counseling experience, and the human-minus-AI difference score. AI-based psychological support. AI-based psychological support received higher ratings on five exploratory item-level indicators, although the response-efficiency comparison should be interpreted cautiously because the item was not strictly measurement-equivalent across modalities. Human counseling received higher ratings on two indicators: mutual understanding and professional competence. The remaining item-level comparisons were not statistically significant after Holm correction. Although the medians were generally high and often identical across modalities, the paired differences indicated a small overall advantage for AI-based support in the primary 11-item comparison. As a sensitivity analysis, the potentially non-equivalent response-efficiency item was removed from both modality-specific scores. The resulting 10-item AI score remained significantly higher than the corresponding human-counseling score, with mean values of 57.77 (SD = 6.55) and 56.80 (SD = 6.75), respectively; mean difference (human minus AI) = −0.97, 95% CI [−1.73, −0.21], t(232) = −2.53, p = .012, dz = 0.17. In hierarchical regression models, attachment anxiety positively predicted both human-counseling and AI-support experience, whereas attachment avoidance negatively predicted both outcomes. Specialized mental health/therapy chatbot use was associated with higher AI-experience scores, while associations involving shorter AI-use duration were less consistent. The human-minus-AI difference-score regression showed improvement in model fit after adding the attachment variables, but the final full model was not significant at the omnibus level. In the repeated-measures mixed-effects analysis, the modality × attachment avoidance interaction was significant, suggesting that the relative difference between AI-based support and human counseling varied according to attachment avoidance. Overall, the sensitivity analysis supported the robustness of the small overall modality difference after excluding the potentially non-equivalent response-efficiency item. Among Chinese adults who had used both AI-based support and human counseling, perceived experiences differed by domain rather than showing a uniform preference for one modality. AI support was evaluated more favorably in low-threshold and low-social-threat domains, whereas human counseling retained perceived advantages in mutual understanding and professional competence. The small overall difference favoring AI persisted after excluding the non-equivalent response-efficiency item, although this result should be interpreted cautiously because the sensitivity analysis used a composite score derived from ordinal item ratings. These findings should be interpreted cautiously because of the cross-sectional self-report design, restricted upper-end variability, heterogeneous self-reported AI tools, and the exploratory nature of the newly developed experience indicators.