Fine-Tuning Large Language Models for Text-to-SQL Tasks in Agricultural Census Anomaly Detection

Detecting anomalies from collected data is essential in producing high-quality data for further analysis, including agricultural census data. This research aimed to improve the efficiency of data anomaly checking by utilizing the CodeLlama- 7B Large Language Model (LLM) to translate human questions into SQL queries. Spider and WikiSQL datasets are used in the first fine-tuning phase for translating ability to general queries. The agricultural datasets are constructed using questions and queries in agricultural census database schemas and then used in the second fine-tuning phase. The LoRA method is implemented in the fine-tuning process. This fine-tuning enhanced the model's performance on the SQL-Eval framework, achieving an Average Correct Rate of 69.6%. Combining this approach with Retrieval-Augmented Generation (RAG) and prompt engineering with Chain of Thought (CoT) improvedthe Average Correct Rate to 77.8%.