An Approach to Detect Large Language Model Generated Firmware for Arduino Platform

In modern education, learning Science, Technology, Engineering, and Mathematics (STEM) has become an essential requirement. Physical computing, embedded systems, and Internet of Things (IoT) have been widely used in STEM education and the Arduino platform has emerged as a de facto standard for teaching and learning them. However, with the recent advancements in Artificial Intelligence (AI), students are tempted to over-relay on Large Language Models (LLMs). As a result, the auto-generation of software source codes by students using LLMs has become a significant threat to achieving the Intended Learning Outcomes (ILOs) of their course of study. Unlike standard software source code developed for general-purpose computing, firmware developed for the Arduino platform directly interacts with hardware components and consists of specific syntaxes. As a result, existing AI-based Auto Code Detectors underperform in detecting auto-generated source code for the Arduino platform. This paper presents an approach for detecting LLM-generated firmware source code for the Arduino platform using code stylometry features. The proposed approach evaluates lexical and syntactic features to differentiate LLM-generated and human-written firmware source codes for the Arduino platform. The results demonstrate a significant mean accuracy of 0.91, indicating that the proposed approach can accurately classify whether code is written by a human or generated by an LLM.