Evaluating AI-mediated Tutors for Competitive-Programming Training: A Position Paper on Prompt-Family Ablation and Rubric Design

Large language models are routinely deployed as tutors in introductory programming (CS1/CS2), and an evaluation literature has emerged around that surface. Competitiveprogramming (CP) olympiad preparation, in contrast, presents a qualitatively different tutoring problem: multi-step algorithmic composition, tight complexity budgets, and adversarial problem framing. No peer-reviewed study evaluates LLM tutors against CP olympiad learners or against a milestone-differentiated cognitive model of CP skill acquisition. This position paper makes three contributions. First, it argues that the CP-tutor evaluation slot is structurally distinct from the CS1-tutor evaluation slot and cannot be filled by extrapolation. Second, it proposes a five-axis rubric (correctness, pedagogical fit, transcript coherence, bias behaviour, Bloom alignment) that extends the four-axis variation schema used in sister work on CP-problem generation. Third, it specifies a three-prompt-family ablation design, one family per knowledge-compilation milestone (L3 strategy retrieval, L4 algorithm adaptation, L5 complexity reasoning), together with a two-channel evidence plan (a Bayesian-Knowledge-Tracing simulated learner and an adult-proxy role-play protocol) that respects a post-pivot admissibility regime excluding K-12 subjects. The paper is positioned as the tutor-side complement to an existing CP-problem-generator methodology, sharing the variation-axis backbone, the pre-registration discipline, and the LLM-as-judge bias-control frame.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-12
DOI
https://doi.org/10.5281/zenodo.20131498
Primary Topic
Teaching and Learning Programming
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Evaluating AI-mediated Tutors for Competitive-Programming Training: A Position Paper on Prompt-Family Ablation and Rubric Design

That Le
Zenodo (CERN European Organization for Nuclear Research)
Teaching and Learning Programming
preprint

Evaluating AI-mediated Tutors for Competitive-Programming Training: A Position Paper on Prompt-Family Ablation and Rubric Design

That Le
preprint en

Abstract

Large language models are routinely deployed as tutors in introductory programming (CS1/CS2), and an evaluation literature has emerged around that surface. Competitiveprogramming (CP) olympiad preparation, in contrast, presents a qualitatively different tutoring problem: multi-step algorithmic composition, tight complexity budgets, and adversarial problem framing. No peer-reviewed study evaluates LLM tutors against CP olympiad learners or against a milestone-differentiated cognitive model of CP skill acquisition. This position paper makes three contributions. First, it argues that the CP-tutor evaluation slot is structurally distinct from the CS1-tutor evaluation slot and cannot be filled by extrapolation. Second, it proposes a five-axis rubric (correctness, pedagogical fit, transcript coherence, bias behaviour, Bloom alignment) that extends the four-axis variation schema used in sister work on CP-problem generation. Third, it specifies a three-prompt-family ablation design, one family per knowledge-compilation milestone (L3 strategy retrieval, L4 algorithm adaptation, L5 complexity reasoning), together with a two-channel evidence plan (a Bayesian-Knowledge-Tracing simulated learner and an adult-proxy role-play protocol) that respects a post-pivot admissibility regime excluding K-12 subjects. The paper is positioned as the tutor-side complement to an existing CP-problem-generator methodology, sharing the variation-axis backbone, the pre-registration discipline, and the LLM-as-judge bias-control frame.

Zenodo (CERN European Organization for Nuclear Research)
ElSohly Laboratories (United States) (US)
Quality Education
Teaching and Learning Programming
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Evaluating AI-mediated Tutors for Competitive-Programming Training: A Position Paper on Prompt-Family Ablation and Rubric Design — That Le · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS