Scorpio: A Verifiable Framework for Enforcing Socratic Scaffolding in Physics LLMs Beyond Fine-Tuning
While Large Language Models (LLMs) have demonstrated significant potential in STEM education, their tendency to provide direct solutions often undermines the learning process. This paper presents Scorpio, a framework that utilizes "Constraint Engineering" to transform a general-purpose LLM into a specialized, verifiable physics tutor. By implementing a layered architecture of inference-time rules, we enforce Socratic scaffolding without the need for expensive fine-tuning. Our results from a 125-response ablation study demonstrate that Scorpio successfully eliminates direct answer delivery (0% DAR), achieving a peak pedagogical quality score of 4.62. Independent expert validation (Ph.D. physics educator) confirmed this success, with the expert independently ranking the FULL constraint system as the highest performing (Expert: 3.83 vs 3.16 baseline) and confirming a significant +0.67 point improvement in pedagogical quality. This work demonstrates that inference-time constraint layering can meaningfully shift LLM tutoring behavior toward Socratic scaffolding, proving that modular constraints can effectively manage complex STEM pedagogy at scale.
Authors
- Rushil Mahadevu (ORCID: https://orcid.org/0009-0001-1213-7851)
Institutions
- Sage (United Kingdom) (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-13
- DOI
- https://doi.org/10.5281/zenodo.21924455
- Primary Topic
- Intelligent Tutoring Systems and Adaptive Learning
- Type
- preprint