A Proof of Concept Demonstration for How to Evaluate Clinical Decision Models

ABSTRACT Decision models are frequently published in applied behavior analysis (ABA) to guide clinical decision‐making, yet few have been empirically tested for their validity. To help researchers and practitioners identify potential weaknesses in decision models, the present study demonstrates one analytic method researchers can use to assess the procedural validity of decision models. Participants ( N = 75) were presented with systematically designed hypothetical vignettes to comprehensively test all decision pathways. Each vignette described a client, challenging behavior, and the targeted therapeutic environment (e.g., school, home) similar to what might be collected during an intake process. Participants were then asked to select the best data collection procedure for their initial behavioral observation either without (baseline) or with access to an associated decision model. Access to the decision model did not significantly improve performance for the control or behavior analyst group. Once these data were collected, we demonstrate how an error analysis allows researchers to identify which decision points incorrect decisions cluster at, providing information on how to improve the language of the decision model. Having an analytic technique that identifies challenges in decision model language and decision paths will aid future researchers in the iterative process i.e. decision model design and validation.