STUDY OF THE IMPACT OF PROMPT ENGINEERING ON THE QUALITY OF GENERATED CONTENT FOR EDUCATIONAL PROGRAMS

This study analyzes how prompt engineering affects the quality of language model–generated content for educational programs. It evaluates various prompting methods across modern models, from compact to large-scale, using a dataset of 8,269 programs. A structural- semantic gradation of components by interpretability was identified: goals, disciplines, and learning outcomes were highly interpretable (high cosine similarity), while components like the Atlas of the profession were less so. Prompting methods showed varying effectiveness: few-shot learning worked best for goals, and chain of thought for professional standards. A key finding is the "specialized advantage": a LoRA-adapted model outperformed larger, non-specialized ones, challenging the idea that bigger always means better. The study offers practical strategies for selecting models and prompts, aiding educational analytics and future research into AI-based interpretation of complex educational content

Authors

Publication Details

Journal
Bulletin of D Serikbayev EKTU
Published
2025-09-30
DOI
https://doi.org/10.51885/1561-4212_2025_3_238
Primary Topic
Open Education and E-Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

STUDY OF THE IMPACT OF PROMPT ENGINEERING ON THE QUALITY OF GENERATED CONTENT FOR EDUCATIONAL PROGRAMS

Andrii O. Biloshchytskyi, Ayanbek Serikov
Bulletin of D Serikbayev EKTU
Open Education and E-Learning
article

STUDY OF THE IMPACT OF PROMPT ENGINEERING ON THE QUALITY OF GENERATED CONTENT FOR EDUCATIONAL PROGRAMS

Andrii O. Biloshchytskyi, Ayanbek Serikov
article en

Abstract

This study analyzes how prompt engineering affects the quality of language model–generated content for educational programs. It evaluates various prompting methods across modern models, from compact to large-scale, using a dataset of 8,269 programs. A structural- semantic gradation of components by interpretability was identified: goals, disciplines, and learning outcomes were highly interpretable (high cosine similarity), while components like the Atlas of the profession were less so. Prompting methods showed varying effectiveness: few-shot learning worked best for goals, and chain of thought for professional standards. A key finding is the "specialized advantage": a LoRA-adapted model outperformed larger, non-specialized ones, challenging the idea that bigger always means better. The study offers practical strategies for selecting models and prompts, aiding educational analytics and future research into AI-based interpretation of complex educational content

Bulletin of D Serikbayev EKTUVol. 109(3)
Openalex Percentile: Top 69%
Open Education and E-Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.