Impacts of Factor Analytic Choices on Scale Interpretation Commonly Used to Assess Children With and Without Autism Spectrum Disorder

Abstract IMPACTS OF FACTOR ANALYTIC CHOICES ON SCALE INTERPRETATION COMMONLY USED TO ASSESS CHILDREN WITH AND WITHOUT AUTISM SPECTRUM DISORDER By Abeera Rehmani Tuesday April 22, 2025 Introduction: Autism Spectrum Disorder (ASD) measures are often designed as diagnostic tools but also used as outcome variables in research. This dual usage poses validation challenges since these tools are typically assessed only for diagnostic accuracy, not as outcome measures. This issue is particularly pronounced for ordinal scales, where their utility as outcome variables is more complex than their validation for clinical diagnostics. Objective: This study examines how various factor analytic decisions influence the quality of scales, using the Childhood Symptom Behavior Questionnaire (CSBQ) as a model, focusing on its development as an outcome variable. Methods: Using responses from approximately 7,000 children in the CDC's Pathway's dataset, this study compares four factor analytic methods—Maximum Likelihood (ML), Categorical ML (cML), Diagonally Weighted Least Squares (DWLS), and Unweighted Least Squares (ULS). These methods are evaluated using fit indices in their regular, scale-shifted, and robust forms to determine their impact on model fit. Additionally, we engaged in the questionable research practice (QRP) of correlating item error terms to “fix” item-factor relationships. Results: Analysis revealed substantial variations in model fit across methods. Scale-shifted and robust conditions notably affected model stability, highlighting the importance of selecting appropriate analytic techniques for ordinal data. Engagement in the QRP of correlating errors to “fix” models was variably required with strongest implications for scale shifted and robust DWLS and ULS models wherein regular unadjusted models did not require fixing but scale-shifted and robust did. Conclusion: The choice of factor analytic method significantly influences scale interpretation accuracy in ASD assessments and conditions eliciting the QRP of “fixing” error terms poorly outlined. This study advocates using methods that accommodate the ordinal nature of data, enhancing the utility of ASD scales in both clinical and public health settings.