B – 147 Utilizing publicly available AI for psychological test coaching

Abstract Objective To examine how publicly available AI can be used to coach patients on common neuropsychological performance validity tests. Method Two common AI search engines (ChatGPT and Grok) were utilized. Five standardized questions were submitted to both search engines to assess the AI’s knowledge of what psychological tests are used to identify malingering, how malingering is identified, and information about the measures, including how to pass them (e.g., cutoff scores). Two separate computers were utilized to examine consistency between search engines. Results Both engines provided in-depth information regarding how psychologists identify and examine malingering. Both search engines emphasized that psychologists use multiple tools to detect malingering, including clinical interviews, self-report questionnaires (e.g., MMPI), standardized psychological assessments (e.g., TOMM), behavioral observations, collateral information, and records review. There were notable differences when looking at details provided and components to answering each question. For example, Grok (xAI) gave more specific information such as listing specific tests that are commonly used (e.g., TOMM, WMT, Rey 15, Dot Counting Test, etc.). Both search engines gave specific cutoff scores when queried, including providing examples. However, Grok provided citations to support cutoff scores, whereas ChatGPT did not. Both AI engines provided statements to not over or under exaggerate symptoms in order to “pass” the performance/symptom validity measures suggesting a programmed “ethics” component. Conclusion Public AI search engines appear to provide accurate information to queries on the specifics of common performance validity measures, including tips and suggestions for how to perform well, or “pass,” validity tests.