Continuous Testing for State-Based Multistep Voice AI Agents

This paper presents a strategy for continuous testing of multi-step AI agents based on finite-state machines (FSM) and integrated with large language models (LLMs). The primary focus is on validating the correctness of transitions (both deterministic and LLM-driven) and evaluating the quality of responses, including testing formats such as voice-to-voice, voice-to-text, and text-to-text. It is demonstrated that the state-based approach, where each FSM node is treated as an independent “module” for testing, allows for error localization and effective application of quality metrics (e.g., G-Eval) in a continuous mode. Additionally, methods for integrating a knowledge base (RAG) into test scenarios and organizing integration and end-to-end (e2e) testing are discussed. The proposed practical recommendations are illustrated through the example of “call booking” and can be extended to more complex voice and text-based dialogue systems.