Generating Medical Diagnostic Scenarios with LLM-Based Reinforcement Learning Feedback: Dataset Release and Methodology

Sample medical scenarios play a crucial role in training healthcare professionals by providing structured cases to develop diagnostic reasoning and clinical decision-making skills. However, access to diverse and inclusive sample diagnostic cases remains challenging due to the limited representation of specific conditions and populations in medical education materials, and existing cases are often not equitable due to a lack of representation of minority groups. In this paper, we present a new dataset of medical diagnostic scenarios generated using a combination of reinforcement learning from artificial intelligence feedback and retrieval augment generation techniques. Despite the dataset’s limited size, it offers a unique resource for advancing medical education, particularly in regions with scarce training materials while also emphasizing inclusivity by incorporating a higher representation of people of color and women. Then, we discuss the data generation process, the dataset structure, and potential applications in medical training programs. This work aims to contribute to the development of accessible, high-quality, and inclusive educational tools in the medical field.