A Generator–Jury Framework for Synthetic Household Financial Narratives
We introduce a reusable generator–jury framework for measuring whether synthetic household fi-nancial narratives support financial assessments consistent with their source records, without requiring human-written reference narratives. The framework uses a two-tier large language model (LLM) jury to derive refer-ence assessments from structured household records. These model-derived reference assessments are producedonce and retained when generation instructions or settings change. Each new narrative is assessed separatelyby the same jury under shared criteria, without access to its source record or reference assessment. Comparingnarrative-derived assessments with the fixed references enables generation conditions to be evaluated on thesame households. The framework measures how closely narrative-derived assessments match assessments ofthe source records, how this agreement varies across household groups, and how often jury responses can beused in the comparison. We applied the framework to baseline and candidate conditions that differed in promptinstructions and thinking mode. The candidate reduced the average gap between narrative-derived and refer-ence financial-health scores by 32%, demonstrating how the framework can guide generation choices duringsynthetic data development.
Authors
- Noorhan Abbas (ORCID: https://orcid.org/0000-0002-1434-6497)
- Feng Miao
Institutions
- University of Leeds (GB)
- University of Cumbria (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24417789
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00