Validity and Reliability of the Athis Multidimensional Attention Performance Measurement System in School-Aged Children: Evidence From an Unselected Reference Sample and Criterion Groups

Abstract Background Attention difficulties are a common reason for referring children for psychological assessment, yet such decisions rest largely on informant ratings. Computerized performance measures provide an observer-independent record but are usually validated against recruited problem-free controls. This study examined the structural validity, reliability, and classification accuracy of the Athis Multidimensional Attention Performance Measurement System, a computerized three-subtest instrument, in children aged 6–11 against an unselected reference sample. Methods The measurement model was estimated in a norm sample of 1,973 Turkish children (split-half replication, confirmatory factor analysis, multi-group invariance). Criterion-related validity was examined in a separately assembled sample of 855 children: 203 with a school-recorded diagnosis of attention-deficit/hyperactivity disorder and no comorbid diagnosis, 97 flagged by teachers but never evaluated, and 555 reported as problem-free, 148 of whom had passed first-stage gifted-education screening. Scores were expressed against age norms from 1,116 unselected children. Classification accuracy was estimated by receiver operating characteristic analysis, cross-validated cut scores, and predictive values at a 7.6% base rate; convergent validity used Conners Teacher Rating Scale scores ( n = 114). Results Each subtest yielded two replicable dimensions, response control deficit and accuracy. Response control deficit separated recorded diagnostic status in all three subtests (AUC = .903, .681, and .698), whereas accuracy, as expected from the construct definition, did not (AUC = .454–.572). Subtest 1 reproduced teacher judgment (AUC = .888), whereas Subtests 2 and 3 separated reported risk from recorded status (AUC = .757 and .750). Accuracy, not response control deficit, responded to selection for cognitive ability ( g = 0.78 versus − 0.22). These estimates are conservative; against a teacher-nominated comparison group, the conventional design, the same contrasts yielded AUC = .992, .813, and .745. At the population base rate the at-least-one-subtest rule had a negative predictive value of 98.0%. Conclusions The system yields an age-referenced two-dimensional profile in each of three tasks and works as a rule-out procedure for population-level screening rather than as an identification instrument. Its accuracy values, estimated against an unselected reference sample, apply directly to unscreened populations, and the comparison-group effect quantified here explains why conventional validation designs report higher values.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-11155561/v1
Primary Topic
Attention Deficit Hyperactivity Disorder
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Validity and Reliability of the Athis Multidimensional Attention Performance Measurement System in School-Aged Children: Evidence From an Unselected Reference Sample and Criterion Groups

Kerem ÇETİNKAYA
Research Square
Attention Deficit Hyperactivity Disorder
preprint

Validity and Reliability of the Athis Multidimensional Attention Performance Measurement System in School-Aged Children: Evidence From an Unselected Reference Sample and Criterion Groups

Kerem ÇETİNKAYA
preprint en

Abstract

Abstract Background Attention difficulties are a common reason for referring children for psychological assessment, yet such decisions rest largely on informant ratings. Computerized performance measures provide an observer-independent record but are usually validated against recruited problem-free controls. This study examined the structural validity, reliability, and classification accuracy of the Athis Multidimensional Attention Performance Measurement System, a computerized three-subtest instrument, in children aged 6–11 against an unselected reference sample. Methods The measurement model was estimated in a norm sample of 1,973 Turkish children (split-half replication, confirmatory factor analysis, multi-group invariance). Criterion-related validity was examined in a separately assembled sample of 855 children: 203 with a school-recorded diagnosis of attention-deficit/hyperactivity disorder and no comorbid diagnosis, 97 flagged by teachers but never evaluated, and 555 reported as problem-free, 148 of whom had passed first-stage gifted-education screening. Scores were expressed against age norms from 1,116 unselected children. Classification accuracy was estimated by receiver operating characteristic analysis, cross-validated cut scores, and predictive values at a 7.6% base rate; convergent validity used Conners Teacher Rating Scale scores ( n = 114). Results Each subtest yielded two replicable dimensions, response control deficit and accuracy. Response control deficit separated recorded diagnostic status in all three subtests (AUC = .903, .681, and .698), whereas accuracy, as expected from the construct definition, did not (AUC = .454–.572). Subtest 1 reproduced teacher judgment (AUC = .888), whereas Subtests 2 and 3 separated reported risk from recorded status (AUC = .757 and .750). Accuracy, not response control deficit, responded to selection for cognitive ability ( g = 0.78 versus − 0.22). These estimates are conservative; against a teacher-nominated comparison group, the conventional design, the same contrasts yielded AUC = .992, .813, and .745. At the population base rate the at-least-one-subtest rule had a negative predictive value of 98.0%. Conclusions The system yields an age-referenced two-dimensional profile in each of three tasks and works as a rule-out procedure for population-level screening rather than as an identification instrument. Its accuracy values, estimated against an unselected reference sample, apply directly to unscreened populations, and the comparison-group effect quantified here explains why conventional validation designs report higher values.

Research Square
Naval Academy (TR)
Türkiye Bilimsel ve Teknolojik Araştırma Kurumu
Good health and well-being
Attention Deficit Hyperactivity Disorder
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon