GMR-159 LLM enabled Synthetic dataset generation for Human-AI teaming Algorithm

This research explores using Large Language Models (LLMs) to generate synthetic datasets for Human-AI teaming algorithms, focusing on mental health assessments. We create a diverse dataset simulating human-AI collaboration scenarios in diagnostic processes. The synthetic data is labeled through an innovative approach involving two human annotators and three LLMs, using majority voting for consensus-based annotations. This dataset serves as a resource for training and evaluating Human- AI teaming algorithms, enabling exploration of collaboration dynamics between human expertise and AI in complex decision-making. Our approach addresses the scarcity of real-world data in Human-AI teaming scenarios and provides a controlled environment for algorithm development, potentially accelerating advancements in this field.