Large Language Models in Data Governance: Multi-source Data Tables Merging

With the explosive growth of data volumes, enterprises face unprecedented challenges in data governance. As a critical step in data modeling, multi-source data table merging enables the integration of multiple tables containing the same entities from different business system databases into a standardized entity table, thereby enhancing data quality and consistency. However, this process relies heavily on manual judgment, leading to inefficiencies. In recent years, the performance of large language models (LLMs) in natural language processing has provided new insights for addressing data governance issues. This paper proposes a "mutation-generation-tuning" approach that enhances the performance of small parameter LLMs (e.g., Baichuan2-7B) in multi-source data table merging tasks through secondary pre-training and fine-tuning. The specific methods include data augmentation and generating intermediate reasoning processes using large parameter LLMs. Experimental results indicate that the optimized MDTM-GPT model exhibits performance on par with high-parameter models in multi-source data table merging tasks while maintaining low cost and data security. The contributions of this paper lie in presenting a novel approach for small parameter LLMs and providing practical validation for applications in the field of data governance.