Quiet threat: Insufficient effort responding in applied linguistics and its impact

Abstract Self‐report surveys are widely used in applied linguistics. Nevertheless, insufficient effort responding—often stemming from a lack of motivation from participants—can compromise survey data quality and distort research findings. This study investigated insufficient effort responding through an online survey assessing second language (L2) teachers’ assessment literacy, employing multiple methods to identify insufficient effort responding: an attention‐check item, self‐reported engagement, response time, and the post hoc index lz . Results indicated that 1.9% to 26.4% of participants exhibited signs of insufficient effort responding, depending on the method used. Insufficient effort responding distorted sample distributions by inflating means and standard deviations and negatively impacted scale reliability and model fit, both of which improved after removing flagged responses. Additionally, insufficient effort responding was less prevalent among older participants, and participants who perceived the survey as “intriguing” or “valuable” reported higher engagement. Furthermore, completing the survey as a favor for friends encouraged participants to put more effort into their responses. Our findings highlight the importance of addressing insufficient effort responding in survey‐based studies. Practical guidelines are provided for improving survey design and administration, emphasizing strategies to prevent and identify insufficient effort responding, thereby enhancing the reliability and validity of self‐report measures in applied linguistics research.