AI Analyst: Safe Write Access and Proposer-Agnostic Visualization Correctness in a Production Multi-Agent Data Analysis System
Large language model (LLM) agents for data analysis have moved from research prototypes to production systems that clean data, query databases, and generate dashboards with little human supervision. As these systems take on more autonomy, two risks that one-shot, read-only designs could previously ignore become load-bearing: first, once an agent (or a non-expert user through an agent-mediated interface) is allowed to write to data, not just query it, a single wrong statement is destructive and the cost of getting it wrong is no longer a bad chart but lost data; second, visualization errors — plotting a row identifier as an axis, aggregating a continuous measure into hundreds of meaningless bins, defaulting to the wrong statistic — occur regardless of whether the chart configuration was proposed by an LLM or by a human, yet existing work frames this as an LLM-specific hallucination problem. We present AI Analyst, a production multi-agent data analysis system, and describe three contributions that address these gaps directly: (1) a transactional write sandbox that lets a non-expert or an agent run arbitrary INSERT/UPDATE/DELETE/DDL safely, using a real database transaction to compute an exact dry-run preview before anything commits, isolated into a working copy that never touches the source data; (2) a proposer-agnostic deterministic correctness gate that every chart configuration passes through identically, whether generated by the agent pipeline, built manually in an interactive editor, or produced by a natural-language edit; and (3) a scheduled, alert-driven monitoring loop that survives a host spinning the service down when idle. We evaluate all three with reproducible, deterministic benchmarks requiring no paid API access: the write sandbox achieves 100% dry-run-to-commit accuracy (7/7) and a false-positive rate of 0/5 on data values that previously triggered a naive keyword blocklist, versus 5/5 before a fix; the visualization pipeline reduces a 199-bar chart to 21 meaningful bins and a 60-category chart to 11, while correctly preserving a 200-point scatter relationship that the pre-fix pipeline collapsed to 199 misleading aggregated points; and the monitoring due-check passes all 12 cases in a hand-verified scenario sweep.
Authors
- Shlok Gupta
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-27
- DOI
- https://doi.org/10.5281/zenodo.23703427
- Primary Topic
- Data Visualization and Analytics
- Type
- preprint