Establishing benchmarks for large language models in nuclear engineering: A NucBench evaluation of GPT-4.1, Claude 3.7 Sonnet, and Gemini 2.5 Pro

• NucBench introduces a nuclear-specific benchmark for multimodal LLM evaluation. • State-of-the-art LLMs are tested on exams and flow-regime images. • Accuracy and stability are quantified under controlled sampling. • Results expose reliability limits in safety – critical nuclear tasks. This study presents NucBench, an open benchmark suite for assessing multimodal Large Language Models (LLMs) in nuclear engineering. NucBench includes curated quantitative, qualitative, and image-based tasks focused on pressurized and boiling water reactors, using standardized textual and visual inputs. As a community-driven resource, it supports consistent evaluation and development of reliable, domain-aware AI tools for reactor design and safety. Three advanced LLMs: GPT-4.1, Claude 3.7 Sonnet, and Gemini 2.5 Pro, were tested on undergraduate and reactor operator exams and two-phase flow image classification. GPT-4.1 achieved the highest quantitative accuracy (up to 95%) under deterministic settings, while Claude 3.7 Sonnet showed the greatest consistency in open-ended tasks. Performance declined and variability increased with higher sampling temperatures, especially for Gemini 2.5 Pro. Image classification accuracy peaked for Churn regimes but fell for complex flows. Results emphasize model tuning, explainability, and benchmark expansion for safety–critical applications.