IRAM-SQL: A Language Model Integrating Relation-Aware Mechanism for Text-to-SQL

The task of text-to-SQL aims to convert natural language questions into executable SQL queries within relational databases. This greatly enhances the usability of relational database querying, alleviating the difficulties faced by users who are not proficient in SQL. A major challenge in text-to-SQL task is the model's ability to recognize entities accurately. This paper focuses on this issue and proposes a language model with an integrated relation-aware mechanism for text-to-SQL, named IRAM-SQL. We explore the adaptability of pretrained models for downstream text-to-SQL tasks, ultimately selecting CodeT5+ as the pretrained model, which is trained on a large dataset of source code and annotation, exploring the shared representation between natural language and code. Therefore, the model's ability to generate structured sentences would be explored during the pretraining phase. At the same time, we also integrate a relation-aware mechanism throughout the model fine-tuning process. To this end, we comprehensively models the relational information between natural language questions and database schemas, integrating these relationships effectively within the encoder and decoder of the Transformer architecture. Two denoising tasks is employed during training to help the model learn these relationships more effectively. The integration of relational information during encoding and decoding, along with the emphasis on entity awareness in the training tasks, enhances the model's entity recognition capabilities. We evaluate our proposed framework on the SPIDER dataset and three of its robustness variants: SPIDER-SYN, SPIDER-DK, and SPIDER-REALISTIC. Experimental results indicate that IRAM-SQL, with its comprehensively integrated relation-aware mechanism, demonstrates strong performance and robustness.