AI-assisted content analysis of a literary illness narrative: a case study in reliability and replication in Peyami Safa’s Dokuzuncu Hariciye Koğuşu (The Ninth External Ward)

Background. Psychological accounts predict a coupling between affective state and mode of reasoning: pain biases processing toward the immediate sensory present, whereas distance from threat supports abstract, comparative and possibility-oriented construal. Extended first-person illness narratives offer a complementary source of description; this study asks what one supports once the reliability of its own scoring is established first. Methods. Each of the 44 chapters of Peyami Safa’s Dokuzuncu Hariciye Koğuşu, a canonical Turkish first-person account of chronic illness under threat of amputation, was characterised for dominant emotional valence and four reasoning modes — metaphorical, analogical, hypothetical and empirical — by AI-assisted first-pass scoring with expert human adjudication. Because that procedure exposes adjudicating readers to the model’s proposals, all 44 chapters were rescored by two further readers working independently and blind to both the model’s candidates and the adjudicated values. Reliability is reported before any substantive result and determines which dimensions carry interpretation. Results. Two of five dimensions met the pre-specified criterion: emotional valence ICC(2,1) = 0.79, 95% CI [0.64, 0.88], and empirical reasoning 0.70 [0.48, 0.83]. Metaphorical (0.50), hypothetical (0.42) and analogical reasoning (0.31) fell below 0.60 and were withdrawn, their values reported as properties of the adjudicated dataset only. The association between the two surviving dimensions replicated: emotional valence and empirical, sensory–bodily content co-varied inversely at ρ = −0.54 blind ( r = − 0.49; −0.57 corrected for unreliability) against ρ = −0.78 adjudicated. The negative affective baseline was reproduced by every score set (32, 39 and 37 of 44 chapters negative). Associations involving the withdrawn dimensions differed substantially across scoring procedures: one changed sign, one increased in magnitude, and one approached zero. Conclusions. The one secure finding is that in this text negative affective tone co-occurs with concrete sensory and bodily representation, and that this survives blind scoring. It is a replicated descriptive pattern, not a measured psychological association; no causal or generalising claim follows. The wider contribution is methodological: AI-assisted, human-adjudicated annotation produced a coherent coefficient structure of which only one association survived blind rescoring, so reliability should be established before substantive interpretation, not after it.