DecisionLens: A Canary Case Study of Request-Validator Misalignment and Fallback in Financial LLM Briefs
Structured generation and application validation can enforce different contracts, making an apparently well-formed financial brief unusable downstream. We examine this failure mode in DecisionLens, a system that combines deterministic financial evidence, structured generation, application validation, bounded corrective retry, and deterministic fallback. A completed Groq Canary crossed four synthetic, 24-month cases with five staged configurations (P0–P4), producing 20 final records from 22 journaled provider attempts. Eighteen attempts returned completed responses and four were recorded as transport failures; 19 finals contain reviewable content and one is unavailable. All four P4 finals used deterministic fallback. Offline replay shows that the three completed P4 responses satisfy the original provider-facing schema but are rejected by the frozen application validator. The request contract permitted free-text focus and action fields, whereas the validator required enumerated values and additional constraints. This establishes a specific integration defect in the tested configuration, not a general estimate of language-model factuality or safeguard efficacy. We document the failure paths, worked financial discrepancies, incomplete annotation provenance, and a subsequently approved but runtime-unverified contract correction. The study offers an auditable account of how schema compliance, application acceptance, output availability, and semantic quality can diverge. Four selected synthetic cases cannot establish comparative superiority, statistical power, or general hallucination reduction. This is an exploratory systems case study of four deliberately selected synthetic cases, not a powered confirmatory study, a peer-reviewed journal article, or evidence of general hallucination reduction. Semantic annotations remain incomplete; no finalized semantic performance rate or independent human-only evaluation is claimed. The held-out Main experiment has not been executed. AI Assistance DisclosureOpenAI Codex and ChatGPT assisted with research software development, offline evidence inspection, preliminary annotation suggestions, and manuscript drafting and editing. Asadullah Hussaini is the sole author and takes responsibility for the research findings, interpretation, and final content. Partial annotations remain classified as AI-Assisted Author Adjudication, not independent human-only evaluation or finalized performance scores. License scope: CC BY 4.0 covers only Asadullah Hussaini's original manuscript, synthetic research data and documentation to the extent of his rights. MIT covers the included original offline verifier and embedded provider schemas. These are scoped mixed licenses, not a blanket license over every ZIP component. Model-output quotations, quoted Groq Support wording, third-party works, dependencies and embedded font programs are excluded from the author's blanket grants; the supplement contains the complete license map and notices. Rights and reproducibility: GroqCloud hosted OpenAI openai/gpt-oss-120b. The three original returned P4 texts, model/provider attribution and content hashes are preserved. Groq Support's author-supplied sanitized guidance generally permits academic redistribution subject to applicable terms; it is not formal legal approval and does not override Beta/model-specific terms or third-party rights. The product-derived canonical validator source is excluded pending specific redistribution-rights confirmation. The public offline verifier checks the sanitized subset but cannot replay that excluded validator by default or independently authenticate withheld private journal bytes.
Authors
- Asadullah Hussaini
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-14
- DOI
- https://doi.org/10.5281/zenodo.23362878
- Primary Topic
- Topic Modeling
- Type
- preprint