Act as an expert in psychometry. The evaluation of large language models utility in psychological tests cross-cultural adaptations

The introduction of Large Language Models (LLMs) has transformed research, particularly in psychometrics, where cross-cultural adaptation of psychological tests presents significant challenges. Traditional adaptations require precise translation and cultural adjustments to ensure measurement validity, making them resource-intensive. LLMs, with their linguistic capabilities, have the potential to streamline this process. However, their risks, including biases and reliability issues, require empirical evaluation. Despite these risks, systematic studies on LLMs' role in test adaptation remain limited. This study assessed the feasibility of adapting psychological tests using LLMs, illustrated by adapting an organizational commitment scale from English to Polish. ChatGPT-4 was employed with a zero-shot learning approach, using structured prompts and psychometric guidelines. The LLM-adapted scale was compared to a human-adapted version through an observational study with 785 participants. The study results indicate that the LLM-adapted scale demonstrated satisfactory psychometric properties, including factor structure, measurement invariance, and convergent and divergent validity, comparable to or exceeding those of the human-adapted version. Although the reliability of the LLM-adapted scale was slightly lower than that of the human-adapted version, it remained within acceptable standards. These findings suggest that LLMs can perform efficient and psychometrically sound test adaptations. This study contributes to the growing discourse on integrating AI into psychological research, offering a scalable method to reduce cultural barriers while emphasizing the importance of oversight to mitigate biases and ensure scientific rigor.