Preserving assistance meaning in cross-source learning analytics
Logs of learners asking an AI system for help are only useful to learning analytics if their meaning survives export and pooling. A request for help, a response the system generated and a hint shown on screen are three different observations, and a missing record does not show that no help was given. I present Learnlog, a draft profile for recording assistance, and a transformer that maps supported native exports into it without asking the source products to change how they log. The profile keeps apart the stage of assistance, what the instrument could capture, what it actually captured, and what is known or unknown, with optional xAPI and Caliper bindings. A reproducible engineering case combines two released datasets from AI-assisted programming environments. Documented native filters keep 1,050 hint-button or retry records and 59 chat-query records. Their source-stratified sum, 1,109, is an inventory of selected released request observations. It doesn't measure a request rate, a dose of assistance or a learning effect. Local validation, carrier round trips and replay checks preserve the selected records and the explicit unknowns. Classifying each request by what the source recorded next shows what pooled request counts hide. In one release, 312 of 1,050 hint requests recorded an error or no display. Pooled with the second release, the request count overstates help recorded as shown by 39%. A third, much larger release, ASSISTments 2012–13, records stages the other two lack. On 86% of its problems with hints, students went on to the last hint, which generally contains the answer. By my episode rules, the system likely broke problems into steps after a wrong first answer 110,513 times. Its emotion columns are detector guesses. Set side by side, the three releases record different subsets of the stages, and only two of them record learners who never asked. Finally, I propose how the same rules could extend beyond assistance to predicting, explaining, revising, transferring, recalling and other learning events. These include a learner's stated confidence and sense of agency, and they're proposals with no results yet. Independent human review of the mappings, validation of upstream capture and any comparison of benefit are still to come. What the report offers is a reusable way to see what assistance records support, and what stays unresolved, before anyone analyzes them across sources.
Authors
- Kumar Sumbhav Srivastava (ORCID: https://orcid.org/0009-0009-5379-4290)
Publication Details
- Journal
- EdArXiv (OSF Preprints)
- Published
- 2026-11-07
- DOI
- https://doi.org/10.35542/osf.io/vs7nf_v1
- Primary Topic
- Online Learning and Analytics
- Type
- preprint