Automated Scoring of Narrative Recall Assessments Using Large Language Models Enables Exploration of Alternate Scoring Criteria

LLMs demonstrate potential as a reliable, cost-effective alternative to human scoring, offering consistent performance and the ability to retrospectively apply modified scoring criteria for research purposes. However, the presence of hallucinations and score variability between model runs suggests that this technology is at a premature stage for use in clinical settings. Nonetheless, rapid advancements in the field may result this strategy being a viable alternative in the very near future, especially as more advanced reasoning models become more available and consistent.