Psychometric evaluation and item reduction analysis of a first-year engineering teaching assistant personality instrument

Abstract Background Personality plays an influential role in educational contexts. Teaching assistants are crucial instructional figures in many undergraduate STEM courses, especially first-year engineering where they have expanded instructional roles. Most educational studies involving personality, however, do not investigate TA personality behaviors. Moreover, an instrument specific for this context that distinguishes between the dual instructor- and student-roles of TAs does not exist. This study addresses this gap, building on previous work that contextualized a Five-Factor Model-based instrument for first-year engineering teaching assistants. Methods The 90-item contextualized First-Year Engineering Teaching Assistant Personality (FYETAP) instrument was distributed as a five-point rating scale from Very Inaccurate (0) to Very Accurate (4) to current and former teaching assistants in first-year engineering programs at four institutions. The resulting data were used for validation analyses to inform an item reduction analysis. The item reduction analysis involved scale-wise Confirmatory Factor Analysis and reliability tests, Classical Test Theory to assess item difficulty and discrimination, and Rasch-based Item Response Theory to analyze facet representation and item overlap. Item reduction was guided by statistical criterion thresholds (factor loadings, reliability changes, discrimination indices, infit/outfit parameters) and conceptual considerations (difficulty range, facet representation, information overlap). Results A total of 196 first-year engineering teaching assistants completed the survey. Overall, items in the Extraversion factor exhibited the best psychometric properties. As such, item reduction for Extraversion was dictated by considerations related to item information overlap with the person density, and facet representation. In the remaining four factors of the scale – Agreeableness, Conscientiousness, Neuroticism, and Openness – about half of the items were removed based on threshold criteria and the other half on considerations of item information overlap and facet representation. Eight items per factor were removed, successfully reducing the instrument to a refined 50-item set. Conclusions This study successfully reduced the contextualized FYETAP to the best 50-items using an item reduction analysis based on a psychometric evaluation process. This instructionally focused tool is critical given TAs’ dual roles as students and instructors, providing a specific instrument to directly inform tailored training, mentoring, and self-reflection. The developed process and resulting instrument offer a unique tool for personality studies in educational psychology.