Reliability and Validity of a Modified Version of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Visit Survey 4.0 (beta) for Ambulatory Behavioral Health Care

Patient experience is a fundamental indicator of healthcare quality. A patient experience survey that assesses telemedicine and in-person ambulatory behavioral health care is needed. This study aims to adapt the CAHPS Clinician and Group Visit Survey 4.0 (beta) to assess adult patient experience with ambulatory behavioral health services and its reliability and validity. Seven behavioral health subject matter experts (psychiatrists, therapists with master’s degrees, and primary care physicians) at an integrated delivery system aligned with an academic institution provided input to select a subset of items and make minor wording adjustments to the CAHPS survey. The adapted survey was administered to 504 adults receiving ambulatory care in a primary care-based behavioral integrated care model. Internal consistency reliability (item-total correlations corrected for item overlap with scale) was 0.49 (0.46) for a 2-item video scale, 0.90 (0.72–0.83) for a 4-item specialist communication scale, and 0.83 (0.84) for a 2-item staff scale. The communication scale had the largest correlation with the overall visit rating (r = 0.86), followed by the staff scale (r = 0.43) and an item about whether the visit started on time (r = 0.41). This study provides initial support for the reliability and construct validity of an adaptation of the CG-CAHPS 4.0 (beta) survey to assess behavioral health care. Providers of ambulatory behavioral healthcare can administer this survey to assess and improve patient experience.