Text-to-SQL Interface using Guardrails and Hallucination Detection

Large Language Models (LLMs) can translate plain-English questions into SQL, which makes institutional databases usable by people who cannot write queries. In a shared system such as a college ERP, however, the generated SQL must be treated as untrusted code: it may modify data, read rows that belong to another user, or quietly answer the wrong question. This paper reports the design, deployment, and security testing of SafeSQL, a natural-language query system over a 25-table, 174- column PostgreSQL ERP schema holding about 200,000 records. SafeSQL places the LLM behind four independent layers: model-level refusal, syntax-tree (AST) blocking with sqlglot, a read-only non-owner database role, and Row-Level Security (RLS). We show that the widely used RLS pattern based on current_setting() can be defeated by generated SQL that calls set_config() from a subquery, CTE, LATERAL join, or window function, and we replace it with a session map keyed on the PostgreSQL backend process, which adds about 2.14 ms per query. We also describe five cases in which our own test infrastructure reported success for the wrong reason, and the checks we added to catch them. The system is deployed as a web application and is covered by 248 automated tests.