Database Quality Assurance with AI : How Large Language models can detect and handle incorrect or anomalous data

Large Language Models (LLMs) have become increasingly common in data-driven processes, including automated data validation. However, important questions remain regarding their reliability, interpretability, and practical applicability in data-critical environments. This study aims to examine how variations in prompt formulation affect the ability of language models to detect errors in a structured relational database. Three different LLMs were evaluated using atotal of 45 prompts, and performance was measured using precision, recall, and F1-score. To analyse the relationship between the semantic similarity of prompts and performance variation, Spearman's rank correlation and principal component analysis (PCA) were applied. The results show that prompts with similar semantic content often yield significantly different performance, while prompts with varied formulations can sometimes produce comparable outcomes. A weak but statistically significant negative correlation was found between semantic distance and the difference in F1-score. This suggests that language models largely respond to surface-level linguistic patterns rather than deeper semantic meaning. Moreover, the PCA visualization revealed no clear clustering of high-performing prompts, further supporting the notion that semantic proximity is not a reliable predictor of model performance. These findings align with previous research emphasizing that even minor variations in syntax and phrasing can greatly impact model output. This study contributes with insights into the limitations of language models in the context of data validation. It also highlights the need for diversity in prompt design and the importance of focusing on linguistic patterning rather than content alone. For future research, it is recommended that the experiment be expanded to include a broader range of LLMs, additional data types, larger prompt sets, and the continued development of effective prompt engineering strategies.