RBDQ: A Reliable LLM-based Text-to-SQL System for Business Data Queries

Using large language models (LLMs) to convert natural language (NL) into SQL simplifies data access for users by allowing them to use everyday language.However, business departments often distrust LLM-based text-to-SQL systems due to the probabilistic nature of SQL generation, which can result in incorrect but executable SQL queries caused by model hallucinations.This leads to significant concerns regarding the accuracy and reliability of the queried data.In this paper, we present RBDQ, a novel LLM-based text-to-SQL system designed to address the unique challenges of business data queries.RBDQ innovatively introduces the Hierarchical Metrics Query Method and integrates advanced Retrieval-Augmented Generation (RAG) methods along with a self-reflection mechanism to tackle these challenges.RBDQ effectively meets the requirements of business metric queries in real-world scenarios.Currently implemented in the Quality Assurance department at ByteDance, RBDQ has significantly improved operational efficiency and query flexibility.Our experiments demonstrate the system's effectiveness, achieving an Execution Accuracy of 96.20%.