(Un)biased data and spin glasses reveal clustering for Turing phase transitions within human-transformer interactions

This paper studies a Large Language Model's ability to exhibit intelligence equivalent to that of a human by analyzing temperature-induced phase transitions, abrupt changes in the macroscopic behavior of a system, in the Turing test. We utilize three approaches: statistical analysis and bias quantification of a human evaluation survey, information retrieval from real human-written versus AI-generated text data using cosine similarity as a comparison metric, and mathematical spin glass model and simulation. We collect text data in the case study of Flitzing, a tradition of emailing poem-like romantic invitations at Dartmouth College because of its richness in information. Across the three approaches, we obtain consistency in phase transition and clustering results, which also align with literature on the mathematics of transformers and metastability. Our work inspires utilizing spin glass theory for the mathematical foundations of artificial intelligence, especially under environmental stochasticity from human interactions, with justification from real data.