Prompt-Condition Variation in Multi-Step Arithmetic Accuracy: A Matched-Task Study Using Llama 3.2 (3B)
This research investigates how prompt-condition variation relates to multi-step arithmetic accuracy in a small language model. The study evaluates Llama 3.2 (3B) using 60 matched arithmetic tasks across three prompt conditions: the original prompt, a rule-based wording and punctuation rewrite, and a whitespace-only control. The experiment contains 180 model responses across operation depths of 16, 18, and 20. The analysis examines overall and depth-specific accuracy, paired differences between prompt conditions, and the stability of model performance under different prompt presentations. The results show an overall accuracy of 86.67% (156/180). No statistically significant differences were found in the primary paired comparisons. The predefined 50% failure boundary was not reached at any tested operation depth and was therefore not estimable. This work provides a controlled, task-matched empirical examination of prompt-condition variation in multi-step arithmetic reasoning. The findings are limited to the tested model, tasks, and prompt conditions.
Authors
- Rajat Pandey
Institutions
- Indian Institute of Technology Madras (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-05
- DOI
- https://doi.org/10.5281/zenodo.23962737
- Primary Topic
- Topic Modeling
- Type
- preprint