Engaging Students in Scientific Writing: The STRaWBERRY Checklist Framework with LLM-based Paper Draft Assessment

Writing scientific papers is essential for advancing any given research field, yet undergraduate and graduate students often struggle with this task, facing challenges in clearly presenting their ideas and results. Valuable scientific contributions described in papers that are not well-structured and not easy to follow may not get published. This paper addresses these challenges by proposing a framework called STRaWBERRY, which provides a checklist-based guide to help evaluate individual components of paper drafts against essential quality criteria. Additionally, we propose the use of Large Language Models (LLMs) to automate the assessment based on these criteria. The LLM evaluation process encourages active learning (in the educational sense) and allows the drafts to be iteratively refined through feedback from the LLM. We evaluate the STRaWBERRY framework by its use in lectures and the corresponding outcome: STRaWBERRY has been successfully used in 10 university courses, leading to multiple student pub-lications. Furthermore, we evaluate the LLM-based approach by assessing its accuracy in evaluating a selection of sample papers, demonstrating its potential to supplement and enhance traditional proofreading cycles.