Assessing Human Evaluations of Cover Letters Written or Edited by AI and Non-Native English Speakers

The assistance of AI plays a significant role in non-native English speakers’ (NNES) formal writing. To investigate the impact of AI on cover letter writing, we designed a survey-embedded experiment. In this study, participants read and evaluated the hireability and writing quality of cover letters created by 4 types of authors respectively: NNES-edited, NNES-written, AI-edited, and AI-written. We ultimately received 118 valid responses from native English Speakers (NES). The results indicated that (1) AI-edited cover letters perform significantly better than other three types of cover letters, but (2) The editing of NNES made the performance of AI-written cover letter worse. Our research provides insights into how artificial intelligence (AI) impacts NNESs with regards to cover letter writing, a critical part of the job application process. This outcome allows us to conclude that NNES would greatly benefit from using AI as an assistive writing tool during their job search.