Knowledge Filtering vs. Architectural Stability: A Comparative Study on Optimizing Lightweight LLMs for Text-to-SQL

To enhance the performance of lightweight large language models (LLMs) on the Text-to-SQL task, this paper proposes and empirically compares two improvement strategies on the Spider benchmark. The first is a knowledge augmentation approach based on Wikidata with relevance-based noise filtering. The second introduces a stabilized cross-attention mechanism that incorporates residual connections and layer normalization. Experimental results show that the latter improves execution accuracy from $81.0 \%$ to $83.3 \%$, achieving a 2.3 percentage point gain, which surpasses the 1.5 point improvement brought by the knowledge-enhanced method. Further analysis reveals that combining the two methods does not yield additive performance gains, suggesting functional overlap between them. This study, built upon lightweight LLMs, highlights that future research on Text-to-SQL should place greater emphasis on architectural optimization of the model core.