Preserving assistance meaning in cross-source learning analytics

Logs of learners asking an AI system for help are only useful to learning analytics if their meaning survives export and pooling. A request for help, a response the system generated and a hint shown on screen are three different observations, and a missing record does not show that no help was given. I present Learnlog, a draft profile for recording assistance, and a transformer that maps supported native exports into it without asking the source products to change how they log. The profile keeps apart the stage of assistance, what the instrument could capture, what it actually captured, and what is known or unknown, with optional xAPI and Caliper bindings. A reproducible engineering case combines two released datasets from AI-assisted programming environments. Documented native filters keep 1,050 hint-button or retry records and 59 chat-query records. Their source-stratified sum, 1,109, is an inventory of selected released request observations. It doesn't measure a request rate, a dose of assistance or a learning effect. Local validation, carrier round trips and replay checks preserve the selected records and the explicit unknowns. Classifying each request by what the source recorded next shows what pooled request counts hide. In one release, 312 of 1,050 hint requests recorded an error or no display. Pooled with the second release, the request count overstates help recorded as shown by 39%. A third, much larger release, ASSISTments 2012–13, records stages the other two lack. On 86% of its problems with hints, students went on to the last hint, which generally contains the answer. By my episode rules, the system likely broke problems into steps after a wrong first answer 110,513 times. Its emotion columns are detector guesses. Set side by side, the three releases record different subsets of the stages, and only two of them record learners who never asked. Finally, I propose how the same rules could extend beyond assistance to predicting, explaining, revising, transferring, recalling and other learning events. These include a learner's stated confidence and sense of agency, and they're proposals with no results yet. Independent human review of the mappings, validation of upstream capture and any comparison of benefit are still to come. What the report offers is a reusable way to see what assistance records support, and what stays unresolved, before anyone analyzes them across sources.

Authors

Publication Details

Journal
EdArXiv (OSF Preprints)
Published
2026-11-07
DOI
https://doi.org/10.35542/osf.io/vs7nf
Primary Topic
Online Learning and Analytics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Preserving assistance meaning in cross-source learning analytics

Kumar Sumbhav Srivastava
2 links
EdArXiv (OSF Preprints)
Online Learning and Analytics
preprint

Preserving assistance meaning in cross-source learning analytics

Kumar Sumbhav Srivastava
preprint en

Abstract

Logs of learners asking an AI system for help are only useful to learning analytics if their meaning survives export and pooling. A request for help, a response the system generated and a hint shown on screen are three different observations, and a missing record does not show that no help was given. I present Learnlog, a draft profile for recording assistance, and a transformer that maps supported native exports into it without asking the source products to change how they log. The profile keeps apart the stage of assistance, what the instrument could capture, what it actually captured, and what is known or unknown, with optional xAPI and Caliper bindings. A reproducible engineering case combines two released datasets from AI-assisted programming environments. Documented native filters keep 1,050 hint-button or retry records and 59 chat-query records. Their source-stratified sum, 1,109, is an inventory of selected released request observations. It doesn't measure a request rate, a dose of assistance or a learning effect. Local validation, carrier round trips and replay checks preserve the selected records and the explicit unknowns. Classifying each request by what the source recorded next shows what pooled request counts hide. In one release, 312 of 1,050 hint requests recorded an error or no display. Pooled with the second release, the request count overstates help recorded as shown by 39%. A third, much larger release, ASSISTments 2012–13, records stages the other two lack. On 86% of its problems with hints, students went on to the last hint, which generally contains the answer. By my episode rules, the system likely broke problems into steps after a wrong first answer 110,513 times. Its emotion columns are detector guesses. Set side by side, the three releases record different subsets of the stages, and only two of them record learners who never asked. Finally, I propose how the same rules could extend beyond assistance to predicting, explaining, revising, transferring, recalling and other learning events. These include a learner's stated confidence and sense of agency, and they're proposals with no results yet. Independent human review of the mappings, validation of upstream capture and any comparison of benefit are still to come. What the report offers is a reusable way to see what assistance records support, and what stays unresolved, before anyone analyzes them across sources.

EdArXiv (OSF Preprints)
Quality education
Online Learning and Analytics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon