Analysis of LLMs for RDF Triple Generation: Semantic and Syntactic Evaluation Using WebNLG

Using the WebNLG dataset as ground truth, we evaluate the capability of large language models (LLMs) to generate resource description framework triples from natural language input. The proposed method employs two complementary evaluation metrics: cosine similarity for assessing semantic proximity and graph edit distance for comparing structural aspects of triple sets. The analysis demonstrates that these metrics provide distinct yet complementary perspectives on semantic evaluation, enabling a comprehensive assessment of the natural language understanding capabilities of LLMs. Through this approach, we demonstrate that modern LLMs exhibit sophisticated abilities in integrated syntax and semantics processing by utilizing distributed representations where both types of information coexist within high-dimensional vector spaces. This integration suggests that understanding of language structure of LLMs transcends simple pattern recognition to achieve meaningful semantic comprehension.