Designing a Serious Game Prototype for Integrated Dialogue Agent Development and First-Aid Research
Responding to a medical emergency requires a bystander to simultaneously call emergency services, communicate observable signs, and perform first-aid actions under the call-taker's guidance. Existing training approaches rehearse these skills separately, and acquired CPR skills decay within months of instruction. We present a serious-game prototype that combines emergency-call practice with first-aid actions in a single interactive scenario. A task-oriented dialogue agent, combining an extended finite-state machine with a large language model (EFSM+LLM), simulates a professional call-taker following established emergency-call protocols. In an office scenario, the player discovers a collapsed colleague and is guided through cardiac arrest detection and CPR. Speech recognition and speech synthesis run locally on the player's device; only transcribed dialogue reaches the dialogue agent, thereby preserving the information asymmetry of real audio-only emergency calls. The resulting system requires the player to coordinate speaking, listening, and acting concurrently, making effective communication part of the task. Its modular architecture separates the dialogue agent, front-end, and dialogue policy. This supports controlled comparisons of interface modality, dialogue policy, and speech processing and provides a reusable platform for research into emergency-call communication and first-aid performance.
Authors
- Jane Jean Kiam (ORCID: https://orcid.org/0000-0002-6830-9931)
- Sean Patrick Klein (ORCID: https://orcid.org/0009-0005-7289-9144)
Institutions
- Universität der Bundeswehr München (DE)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-29
- DOI
- https://doi.org/10.5281/zenodo.23775150
- Primary Topic
- Speech and dialogue systems
- Type
- preprint
Funders
- European Commission