Lightweight Language Models in Autonomous Systems: In-Context Learning with Bidirectional Alignment for Fault Detection

Large language models (LLMs) have shown remarkable capabilities in generalization through in-context learning (ICL) examples. However, the computational demands of LLMs have driven the development of lightweight language models (LightLMs), which are more efficient for specialized applications such as unmanned aerial vehicles (UAVs), autonomous vehicles, and robotics. This paper proposes a framework for fault detection in autonomous systems using pre-trained LightLMs with an ICL approach enhanced by bidirectional alignment (BiAlign). BiAlign is employed to optimize the selection and alignment of ICL examples, aligning both input preferences and output patterns. This method effectively enhances the pretrained LightLMs' ability to process examples without requiring additional training or fine-tuning. We validate our framework using the real-world ALFA dataset of UAVs, evaluating its performance based on accuracy, precision, recall, and F1 score. Moreover, our framework shows significant potential in executing tasks related to autonomous systems. It achieves high evaluation metrics for fault detection using pre-trained LightLMs with only 10 ICL examples.