Impact of Large-Model Temperature on Generative Relation Extraction Results

This paper investigates the impact of temperature parameters in large language models (LLMs) on relation extraction. Through experimental studies, we analyze the performance of LLMs in terms of accuracy, completeness, and consistency for relation extraction tasks under different temperature settings. The experiments show that, in terms of accuracy, at low temperatures (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T&lt;1$</tex>), accuracy first increases and then decreases as the temperature rises. In contrast, at high temperatures (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T \geq 1$</tex>), accuracy decreases as the temperature increases. At low temperatures, relation extraction demonstrates high precision for common relation types but suffers from low recall for rare relations. As the temperature increases, the recall rate for rare relations improves; however, overall accuracy declines due to an increase in misjudgments. Studying the performance of different model parameters at varying temperatures reveals the synergistic effect between “version iteration optimization” and “parameter scale dividend” in LLMs. Regarding model versions, Llama3.1-70b, and Llama3-70b exhibit better accuracy and completeness at low temperatures (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T&lt;1$</tex>), while Llama3-70b shows the best performance in these aspects at high temperatures (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T \geq 1$</tex>). In terms of model parameters, increasing the parameter scale yields higher task accuracy, smaller fluctuations in the F1 score, and more stable performance. For consistency in multi-round extraction, low temperatures yield results with high similarity and strong consistency but weak capability to capture dynamic information. In contrast, high temperatures result in low similarity and poor consistency.