A Conversational Multi‑Agent Framework for Prompt Evaluation across Large Language Models

Large Language Models (LLMs) are increasingly deployed in diverse applications, yet designing effective prompts that generalize across multiple LLMs remains challenging.This paper proposes a conversational multi-agent framework for testing and evaluating AI prompts using multiple LLMs (ChatGPT, Claude, Google Gemini) in a collaborative setup.The framework introduces a multi-agent architecture where AI agents powered by different LLMs interact under an orchestrator to process user prompts and evaluate responses collaboratively.A dynamic conversational interface enables prompt refinement and testing in real-time, providing immediate feedback on prompt efficacy.Key evaluation metrics include fluency, task success rate, response diversity, coherence, and groundedness to systematically assess prompt outcomes.Comprehensive experiments across 12 diverse datasets and 8 prompt categories demonstrate that multi-LLM collaboration surfaces strengths and weaknesses of prompts more effectively than single-model testing, with statistical significance (p<0.05).This work contributes a novel interactive approach to prompt engineering by leveraging multi-agent conversations to ensure prompts elicit high-quality, coherent, and factual responses across leading LLMs.