SoraniSQL: A Novel Synthetic Text-to-SQL Dataset and Benchmark for Low-Resource Central Kurdish (Sorani)

Abstract Text-to-SQL systems allow users to query databases in natural language, yet existing benchmarks cover English and a small number of well-resourced languages, and no resource has been reported for Kurdish. This paper presents SoraniSQL, a novel dataset and benchmark that is, to the best of current knowledge, the first for Text-to-SQL in Central Kurdish (Sorani). The dataset contains 7,518 question-SQL pairs from 609 query types, systematically constructed over a nine-table retail point-of-sale database with four database instances for test-suite execution accuracy (EXₜₛ); a five-member native-speaker panel rated a stratified sample of 200 questions at 4.9 out of 5 for fluency and 4.9 for adequacy. Dedicated splits separate new values, unseen phrasings, unseen combinations of query components, unseen SQL constructs and spelling variation. Experiments with retrieval, template, compositional, neural and pretrained models, all tuned on development data, show that a template classifier reaches 97.1% EXₜₛ on new values but 0.7% on unseen combinations, where a compositional model reaches 97.7%. A 1.5B-parameter code model fine-tuned with quantised low-rank adaptation (QLoRA) reaches 95.0% on unseen combinations on a laptop central processing unit (CPU), and the structure split and the orthographic-noise sets form the most demanding conditions of the benchmark. A Windows application, DanaText2SQL, demonstrates offline use of the benchmark's best fast model with Microsoft SQL Server.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-13
DOI
https://doi.org/10.21203/rs.3.rs-11715844/v1
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

SoraniSQL: A Novel Synthetic Text-to-SQL Dataset and Benchmark for Low-Resource Central Kurdish (Sorani)

Dana Lattef Hussein
2 links
Research Square
Natural Language Processing Techniques
preprint

SoraniSQL: A Novel Synthetic Text-to-SQL Dataset and Benchmark for Low-Resource Central Kurdish (Sorani)

Dana Lattef Hussein
preprint en

Abstract

Abstract Text-to-SQL systems allow users to query databases in natural language, yet existing benchmarks cover English and a small number of well-resourced languages, and no resource has been reported for Kurdish. This paper presents SoraniSQL, a novel dataset and benchmark that is, to the best of current knowledge, the first for Text-to-SQL in Central Kurdish (Sorani). The dataset contains 7,518 question-SQL pairs from 609 query types, systematically constructed over a nine-table retail point-of-sale database with four database instances for test-suite execution accuracy (EXₜₛ); a five-member native-speaker panel rated a stratified sample of 200 questions at 4.9 out of 5 for fluency and 4.9 for adequacy. Dedicated splits separate new values, unseen phrasings, unseen combinations of query components, unseen SQL constructs and spelling variation. Experiments with retrieval, template, compositional, neural and pretrained models, all tuned on development data, show that a template classifier reaches 97.1% EXₜₛ on new values but 0.7% on unseen combinations, where a compositional model reaches 97.7%. A 1.5B-parameter code model fine-tuned with quantised low-rank adaptation (QLoRA) reaches 95.0% on unseen combinations on a laptop central processing unit (CPU), and the structure split and the orthographic-noise sets form the most demanding conditions of the benchmark. A Windows application, DanaText2SQL, demonstrates offline use of the benchmark's best fast model with Microsoft SQL Server.

Research Square
Sulaimani Polytechnic University (IQ)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon