Inter-rate Reliability Of Mental Toughness Scores By Head And Assistant Coaches: A Repeated-measure Design

Mental toughness (MT) is a crucial attribute in sports, often linked to superior performance. Female sports such as rowing are underrepresented in MT research. In addition, MT research has primarily relied on cross-sectional self-assessment, raising questions about the reliability of scores. Recently, external evaluation (e.g., supervisors, peers, experts) has been proposed as an alternative method of MT assessment. PURPOSE: To investigate the inter-rater reliability (IRR) of MT assessments provided by the head coach (HC) and the assistant coach (AC) in the context of collegiate female rowing, using a repeated-measure design. METHODS: Twelve Division II female rowers (Age: 20.8 ± 2.1 years, mean±(SD)) participated in the study. MT was assessed by both HC and AC using the Mental Toughness Index (MTI) integrated into their periodization program. Data collection spanned three months, with MTI completed before each of the four in-season 2K races. IRR between HC and AC regarding average MTI scores was calculated using the interclass correlation coefficient (ICC) in the R statistical package irr, within Jamovi version 2.4.8 (p < 0.05). The two-way mixed model, agreement, average-measures ICC was employed to assess the level of agreement between HC and AC in their MT scores across these four occasions. RESULTS: The resulting ICC (0.83) fell within the good reliability range (0.75 ≤ ICC < 0.90) (F11,10.6 = 6.74, p < .001, 95%CI [.44, .95]), indicating substantial agreement between HC and AC in their MT assessments, suggesting that MT was consistently rated across the coaches. CONCLUSION: This investigation highlights the robust reliability of the MTI scores across divergent coaching evaluations. These findings constitute strong evidence for the psychometrically-sound properties of MTI, affirming its capacity in capturing the latent MT construct within this athletic cohort. As such, practitioners, both in the realms of sport psychology and coaching, can readily embrace MTI as an instrument for MT assessment.