Virtual Tools for Developing SCLAs: An Example of Concentration Analysis

Standardized Conceptual Learning Assessments (SCLAs) measure individuals’ conceptual understanding using distractor-driven multiple-choice items. Concentration analysis (CA), a quantitative technique that evaluates the concentrations of each item’s response distribution allowing researchers to determine how many models are triggered by each item, and can be utilized to help establish the validity of distractor driven items. As part of the development of the Fluids Conceptual Evaluation (FCE), a distractor-driven two-tier multiple-choice evaluation instrument, the authors evaluated the concentration factors of a set of responses from the FCE’s preliminary pilot test results. Results of this analysis show how many different mental models are triggered by each item’s first tier. This paper demonstrates how the use of CA verifies recommended models, and suggests a similar model structure for concentration deviation. Recommendations for future use of concentration analysis will be made accordingly. Supported by NSF Award # 2021059 (AAPT), 2021261 (UNE) and 2021273 (UNH).