Sociocognitive Theory Implications for the Assessment on the AI (AoA) Framework

Generative-AI agents now write assessment tasks, deliver them, score responses and phrase interpretations, so the argument that licenses a score interpretation passes through components that are neither the test taker nor a fixed instrument. This paper asks where, in such a system, the sociocognitive argument about what a performance means has to be made. Drawing on evidence-centered design, argument-based validity and Mislevy's sociocognitive account of the fit between situation features and person-level resources, it argues that each role an agent occupies is a boundary across which one inference of the assessment argument is carried, and that the sociocognitive conditions of that inference have to be argued at the boundary as well as at the person. For task generation, delivery, evidence identification, evidence accumulation, interpretation and monitoring, it states the inference carried and the condition to be argued. The Assessment on the AI (AoA) framework, a multi-agent system built by the first author, is described as one architecture in these terms: its agents exchange addressed ontology models, so what crossed a boundary can be retrieved and inspected, and human checkpoints place interpretive decisions with educators. The paper separates what is built from what is specified and from what it argues. Implications for generated tasks, for educator authority and for the validation of systems of boundary inferences are stated as properties and open questions, not as outcomes, and a research agenda turns each into an empirical question. No outcome data are reported, and the inspectability of a boundary is distinguished from the correctness of the inference carried across it.

Authors

Publication Details

Journal
EdArXiv (OSF Preprints)
Published
2026-11-07
DOI
https://doi.org/10.35542/osf.io/hdf78_v1
Primary Topic
Intelligent Tutoring Systems and Adaptive Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Sociocognitive Theory Implications for the Assessment on the AI (AoA) Framework

Rachel Y. Choi, Jaehwa Choi
3 links
EdArXiv (OSF Preprints)
Intelligent Tutoring Systems and Adaptive Learning
preprint

Sociocognitive Theory Implications for the Assessment on the AI (AoA) Framework

Rachel Y. Choi, Jaehwa Choi
preprint en

Abstract

Generative-AI agents now write assessment tasks, deliver them, score responses and phrase interpretations, so the argument that licenses a score interpretation passes through components that are neither the test taker nor a fixed instrument. This paper asks where, in such a system, the sociocognitive argument about what a performance means has to be made. Drawing on evidence-centered design, argument-based validity and Mislevy's sociocognitive account of the fit between situation features and person-level resources, it argues that each role an agent occupies is a boundary across which one inference of the assessment argument is carried, and that the sociocognitive conditions of that inference have to be argued at the boundary as well as at the person. For task generation, delivery, evidence identification, evidence accumulation, interpretation and monitoring, it states the inference carried and the condition to be argued. The Assessment on the AI (AoA) framework, a multi-agent system built by the first author, is described as one architecture in these terms: its agents exchange addressed ontology models, so what crossed a boundary can be retrieved and inspected, and human checkpoints place interpretive decisions with educators. The paper separates what is built from what is specified and from what it argues. Implications for generated tasks, for educator authority and for the validation of systems of boundary inferences are stated as properties and open questions, not as outcomes, and a research agenda turns each into an empirical question. No outcome data are reported, and the inspectability of a boundary is distinguished from the correctness of the inference carried across it.

EdArXiv (OSF Preprints)
Intelligent Tutoring Systems and Adaptive Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon