COGNAC at CQs-Gen 2025: Generating Critical Questions with LLM-Assisted Prompting and Multiple RAG Variants

We describe three approaches to solving the Critical Questions Generation Shared Task at ArgMining 2025.The task objective is to automatically generate critical questions that challenge the strength, validity, and credibility of a given argumentative text.The task dataset comprises debate statements ("interventions") annotated with a list of named argumentation schemes and associated with a set of critical questions (CQs).Our three Retrieval-Augmented Generation (RAG)-based approaches used in-context example selection based on (1) embedding the intervention, (2) embedding the intervention plus manually curated argumentation scheme descriptions as supplementary context, and (3) embedding the intervention plus a selection of associated CQs and argumentation scheme descriptions.We developed the prompt templates through GPT-4oassisted analysis of patterns in validation data and the task-specific evaluation guideline.All three of our submitted systems outperformed the official baselines (0.44 and 0.53) with automatically computed accuracies of 0.62, 0.58, and 0.61, respectively, on the test data, with our first method securing the 2nd place in the competition (0.63 manual evaluation).Our results highlight the efficacy of LLM-assisted prompt development and RAG-enhanced generation in crafting contextually relevant critical questions for argument analysis.