Use of a Large Language Model to Reveal Narrative Architectures of Veteran Transition Stress: Development and Validation Study

Background: The stress caused by multiple aspects of veterans' transitions from military to civilian, termed transition stress, represents a unique source of psychological impact that is underresearched due to its qualitative nature. The assessment of this complex psychological phenomena has thus relied on laborious interviews designed to extract quantitative information from qualitative narratives of the transition to civilian life. We sought to determine if large language models (LLMs) could be used as valid measurement tools to extract relevant information from open-ended narratives. Objective: This study sought to develop and validate a generative artificial intelligence (AI) approach to automate the quantification and subsequent thematic analysis of veteran transition stress. Methods: Utilizing transcripts from interviews of a sample of US military veterans, we developed an LLM to rate transition stress severity and examined the model's reliability in relation to human coders and validity in relation to a set of related questionnaire measures. Next, we used the LLM scores to quantitatively define high and low transition stress groups, enabling a targeted, automated analysis of themes related to narrative identity and life transition themes that might differentiate the two groups. Results: LLM ratings of transition stress correlated highly with the human expert ratings and showed significant, theoretically congruent correlations with measures of clinical symptoms, reintegration difficulties, and veterans' self-ratings of transition difficulty. Critically, the AI-derived thematic analyses of the narratives from high and low transition stress veterans revealed clearly distinct and informative patterns. Conclusions: These findings suggest that generative AI offers a robust, scalable, and reliable method for multidimensional analysis of complex, narrative-based psychological constructs.