ATDLLMD: a test-driven framework for safe, reliable, and business-centric LLM development

Rapid progress in the field of large language models (LLMs) has enabled a path to a wide array of innovative applications, but this progress has created new challenges, such as reliability, safety, and alignment with evolving business needs. So we introduce ATDLLMD, an Acceptance Test-Driven LLM Development framework that integrates agile best practices, data-centric methods, and safety engineering into the LLM development process. Addition of an iterative and user-centric feedback loop to traditional acceptance test-driven development (ATDD) enables ATDLLMD to continuously refine both the model and its surrounding systems. Central to our approach are two innovations: the adaptation of the Cognitive Process Management for AI (CPMAI) framework to drive agile, test-first development, and the introduction of LM-Eval—a bespoke evaluation tool that enables business-centric and safety-aware testing. Applied to the MediVoice use-case—a phone bot that autonomously manages patient services—our methodology demonstrates how iterative red-train-green cycles can enhance model performance, build user trust, and ensure adherence to safety specifications via the Guaranteed Safe AI framework. Results from this approach illustrate that ATDLLMD not only streamlines LLM development but also provides a robust roadmap for deploying LLM-based applications in safety-critical environments.