Do Large Language Models Require Prior Knowledge for Learning? A Preliminary Study

LLMs find increasing use as an educational tool, both from the instructors' perspective as well as from the students'. This paper presents a preliminary study investigating the effects of prior knowledge concerning a given topic on the effectiveness of using an LLM (Microsoft Copilot) to study the topic. Choosing Büchi automata as an example, twelve computer science students were tasked with first giving a self-report on prior knowledge, then studying Büchi automata for 15 minutes using only an LLM as a study tool, and afterwards filling out a short topical questionnaire. Two trends could be observed: Prior knowledge of LLMs seems to increase the learning effect while prior knowledge of a topic that is related to the studied topic seems to diminish the learning effect.