Case reports unlocked: Leveraging retrieval-augmented generation with large language models to advance research on psychological child maltreatment

RAG based assessment can replicate human judgment even on complex CPS case factors. High accuracy and complete inter-rater agreement level was achieved for factors that are straightforward to classify, such as parental alcohol abuse. The effectiveness of these methods stems from the presence of contextual clues related to case factors within a few sentences across different sections of the text, rather than from characteristics inherent to the entire text. For case factors such as parental lack of cooperation, both supporting and refuting evidence needs to be assessed to achieve optimal accuracy. Careful consideration of potential biases and limitations in RAG methods is advised. These applications can serve as early warning systems, by identifying critical factors from extensive case notes that might otherwise be overlooked, supporting professionals in making informed decisions and improving outcomes for at-risk children.