Assessing Zero-Shot and Zero-Shot Chain-of-Thought Reasoning Abilities in JAMB Mathematics and Physics Exams: Do LLMs 'Know' JAMB?

In this study, we investigate the zero-shot and zero-shot chain-of-thought reasoning capabilities of advanced language models GPT-4, Claude and Mistral on the Joint Admissions and Matriculation Board (JAMB) Mathematics and Physics examinations.The evaluation focuses on the models' inherent ability to solve standardised test questions that require logical reasoning and domain-specific knowledge, without any prior fine-tuning.Past JAMB exam questions were systematically presented to each model in both zero-shot and chain-of-thought prompting conditions.We analysed performance using metrics such as accuracy, reasoning quality, response time and computational efficiency to draw comparisons across models and evaluation methods.The findings reveal the strengths and limitations of each model in tackling complex problem-solving tasks, with particular emphasis on the impact of chain-of-thought prompting on reasoning performance.These results provide valuable insights into the potential of large language models in educational contexts, particularly in the development of automated tutoring and assessment tools.The study also identifies areas where current models perform well and where further improvement is needed, offering guidance for future research and AI system design tailored to Mathematics and Physics education within the Nigerian curriculum.