Embedded and Real-Time Anomalous Command Classification in Unmanned Ground Vehicle Operations

Unmanned Ground Vehicles (UGVs) are increasingly used in safety-critical applications, typically controlled under challenging conditions that may cause stress and fatigue on the operator. This can potentially compromise the safety of the mission due to involuntary movements by the operator, resulting in abnormal commands issued on the remote controller of the vehicle. Such movements can be detected by evaluating the mental state of the operator, the operational context of the UGV, and of course real-time movement detection by the operator. To detect such anomalous commands, we propose a three-stage Machine Learning (ML) based approach, which is suitable for embedded implementation providing real-time classification. Firstly, we detect whether there is any movement on the controller and classify the type of the given movement. Next, we classify whether the operator is under incremental stress and finally discern, in relationship to the UGV's operational context extracted by the state of the UGV is considered normal or not. We use a dataset collected through real-world scenarios, to evaluate the proposed approach, and we evaluate the performance of the proposed approach on an embedded platform (Jetson Xavier NX), extracting relevant metrics such as processing time, energy consumption, and classification accuracy. Our findings demonstrate successful high classification rates for the mental state of the operator (96%) and the ground vehicle movements (74%), as well as the associated involuntary command recognition, with low energy <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(0.43mJ)$</tex> and time <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(0.39s)$</tex> requirements.