Once is enough! An analogue study on repeated validity assessment in adults with ADHD

Performance validity tests (PVTs) can be seen as gatekeepers for valid neuropsychological assessment, by marking cognitive test scores that may not reflect true ability levels. The present study explored the significance of repeated validity testing of adults with attention-deficit/hyperactivity disorder (ADHD), by exploring the potential value of performance consistency across assessments. The operational definition of performance consistency was determined by calculating the mean variation in a participant's PVT scores across three separate assessments. Neuropsychological test data of 24 individuals diagnosed with ADHD were complemented by an analogue study involving 69 typically developing individuals who were allocated to either a control group or a simulation group instructed to feign ADHD. All individuals were assessed with embedded and stand-alone PVTs three times with one-month intervals between each assessment. The rate of failed validity testing remained rather stable across assessments. Significant differences in neuropsychological performance scores occurred between individuals with ADHD and experimental simulators, however, mostly nonsignificant effects of small size emerged when considering performance consistency. Our data demonstrate that the consistency of cognitive performance over repeated assessments may be no effective approach to complement validity assessment. Replication is needed in independent research on larger samples.