Operating Conversational Large Language Models (LLMs)in the Presenceof Errors

Conversational Large Language Models have taken the center stage of the artificial intelligence landscape. As they are pervasive, there is a need to evaluate their dependability, i.e., performance when errors appear due to the underlying hardware implementation. In this paper we consider the evaluation of the dependability of a widely used conversational LLM: Mistral-7B. Error injection is conducted, and the Multitask Language Understanding (MMLU) benchmark is used to evaluate the impact on performance. The drop in the percentage of correct answers due to errors is analyzed and the results provide interesting insights: Mistral-7B has a large intrinsic tolerance to errors even at high bit error rates. This opens the door to the use of nanotechnologies that trade-off errors for energy dissipation and complexity to further improve the LLM implementation. Also, the error tolerance is larger for 8-bit quantization than for 4-bit quantization, so suggesting that there will be also a trade-off between quantization optimizations to reduce memory requirements and error tolerance. In addition, we also show the different impact of errors on different types of weights, which is valuable information for selective protection designs.