DecisionTraceBench: Tracking Decision-Critical Information in Multi-Party Spoken Conversations
In high-stakes domains such as finance, critical information evolves through multi-party interaction. Recovering these changes and their sources is fundamental to reliable speech understanding. Existing speech benchmarks focus on transcription, local understanding, and turn-level state prediction, leaving joint decision evolution across speakers insufficiently evaluated. We introduce DecisionTraceBench, a speech state-tracking benchmark for Decision-Critical Information State Tracking (DCIST). It jointly evaluates decision event sequences, speaker attribution, scope, and temporal evidence, using trajectory-level F1 to measure complete event-chain recovery. Our dataset comprises Chinese financial dialogues involving two to four speakers, with 100.39 hours of synthetic audio and 8.61 hours of human-performed recordings. On a fixed cohort of human-performed recordings, five cascaded systems and two direct-audio models achieve DCIST scores of 5.73% to 21.35%. Ablations with speaker-labeled reference scripts and no temporal localization requirements yield a maximum score of only 38.72%. Audio perturbation experiments further show that overall transcription error rates do not adequately reflect complete trajectory recovery. These findings reveal limitations in jointly understanding information provenance and state evolution, providing a benchmark for evaluating decision-process understanding and decision traceability in high-stakes speech applications.
Authors
- Fangxin Shang (ORCID: https://orcid.org/0000-0002-9659-0134)
- Wenchao Du (ORCID: https://orcid.org/0000-0001-9237-7048)
- Runze Cui
- Yehui Yang
- Long Wu
- Qing Yang
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-10-26
- DOI
- https://doi.org/10.48550/arxiv.2610.34746
- Primary Topic
- Speech and dialogue systems
- Type
- preprint