What do officers learn from an AI-simulated suspect? An exploratory pre/post pilot of police questioning for a suspect with developmental language disorder

Abstract Background People with speech, language and communication needs, including undiagnosed developmental language disorder (DLD), are over-represented in the criminal justice system, yet police questioning is rarely adapted to them. Large language models make simulated interview practice possible, but the effects of such training, and the accuracy of the automated scoring that supports it, are largely unknown. We evaluated the Forensic Interview Trainer, in which a simulated 19-year-old suspect with undiagnosed DLD responds through a large language model constrained by deterministic rules, and a rule-based auditor scores each officer’s question against five communication strategies. Results Eighteen serving police officers interviewed the simulated suspect for five minutes, viewed a short briefing, and interviewed him again, and 927 questions were compared after three systematic errors in the auditor had been corrected. The proportion of questions classified as Recommended rose from 28.2% to 46.1% (mean within-officer change 18.0 percentage points, 95% confidence interval 8.1 to 31.0), and 15 of 18 officers improved, with fewer closed questions, difficult words and multi-part questions, and fewer content words per question. The original auditor misclassified 22% of questions, always under-rating the officer, and because its flags also drove the simulated suspect, 12% of questions received an automatic refusal triggered by an everyday word, a name or a transcription error. In exploratory analyses, questions opening with fillers such as “okay” fell from 51.1% to 17.6%, although the briefing never mentioned them. Officers rated the training highly, and several noticed the scoring errors, yet all agreed that the feedback was useful. When the suspect gave a non-answer, officers tended to narrow to closed questions or to challenge him rather than simplify. Conclusions A short briefing combined with practice against a language model-simulated suspect was associated with a large improvement in officers’ questioning of a suspect with communication needs. However, errors in the automated scoring shaped both the simulated suspect’s replies and the scores officers were shown, and the uncontrolled design cannot isolate the contribution of the simulation. Automated scoring in AI-mediated training should be validated against expert judgement before its feedback is used to teach.