Advanced Question Answer Generation (QAG) for mastering vocabulary proficiency using Generative AI

Question Answer Generation (QAG) is an essential task in Natural Language Processing (NLP) and Artificial Intelligence (AI). QAG has widespread applications, particularly in the education domain such as, creating content for educational tools and tutoring systems, as well as generating questions for the Scholastic Assessment Test (SAT). Currently, creating such content is largely manual, time-intensive, and is reliant on human expertise. This research focuses on evaluating Meta’s Llama3 for generating Words in Context questions which is one of the categories in the SAT’s reading and writing section under the Craft and Structure section. The research demonstrates how the Llama 3 8B model can be leveraged to generate context-relevant SAT questions based on a given genre and word combination. Specifically, In-Context Learning (Prompt Engineering) techniques along with Semantic Similarity (SS) embedding models are used as part of the experiments. Baseline test against the model resulted in a low accuracy of 47%. After experimenting with different prompt engineering techniques and dataset fine tuning using SS, Retrieval Augmented Generation (RAG) yielded better results with 74% accuracy. This research shows that tutors can leverage RAG + SS technique to generate Word in Context questions in an automated fashion and thereby reducing the manual effort in the generation part. Evaluation requires human assessment but that is aided by automated reasoning. Codebase that resulted from this research can be used to generate such questions for any genre and word combination, and delivered through a mobile/web-based application to help students master SAT vocabulary. The approach used allows for genre input from various categories, such as science and history; among others; to generate creative content. This research paves the way for leveraging Gen AI for other Reading/Writing SAT sections.