Exploring AI Bots as Simulators in Human Subject Research: A Novel Approach to Ethical and Efficient Experimentation in Engineering Education Research

This research paper aimed to evaluate the effectiveness of AI bots in simulating human subjects for research purposes. In recent years, the use of artificial intelligence (AI) in education has gained significant attention. While most approaches explore possibilities for teaching and learning, the possibilities for human subject research have been under-explored. By utilizing AI bots as a tool for data collection and analysis, researchers can potentially reduce costs, increase efficiency, and improve overall accuracy in their studies. This project focused on replicating a traditional qualitative research study on the topic of “humility in engineering“ with real human subjects in a simulated study using several publicly available generative AI tools - ChatGPT, Gemini, DebateAI, MasterDebater - to evaluate the effectiveness of AI bots as simulators of human subject research. The project assessed the “personalities“ and response patterns of AI tools leading to ethical implications of using AI bots as research simulators. In addition to evaluating the effectiveness and ethical implications of AI bots as simulators of human subject research, the project also explored the potential benefits and limitations of this approach. Overall, this project represents an important step towards understanding the role of AI bots in research and their potential impact on the engineering education research community. By evaluating the use of AI bots as simulators of human subject research, researchers can gain valuable insights into the benefits, limitations, and ethical considerations of this approach. Ultimately, this project seeks to advance the field of research methodology and contribute to the ongoing dialogue surrounding the use of AI in scientific studies.