Quantitative Analysis of Generative AI Text Usage and Identification of Factors Influencing Text Choice

Abstract As tools that produce AI-generated text evolve rapidly and increasingly integrate into our everyday lives, it is essential to understand how they are used and what factors influence people to include AI-generated text in their written work. To directly assess the use of text produced by generative-AI tools and to begin identifying factors that may influence its use, we quantitatively measured, at the individual-word level, the percentage of human- and AI-generated text in undergraduate students’ essays. These results produced a dataset used to identify factors influencing students' use of text generated by generative AI. Our analysis of the percentage of human- and AI-generated text across over 300 student essays revealed that usage of AI-generated text in a student’s essay varied by major, area of study, and a student's academic ranking in college. These data demonstrate the feasibility of tracking students’ use of generative AI in written work, identifying factors influencing how AI-generated content is used, and beginning to provide insights into how students choose to incorporate it into their essays.