Model Selection and Prompting Strategies in Resource Constrained Environments for LLM-based robotic system

A presentation about article: "Model Selection and Prompting Strategies in Resource-Constrained Environments for LLM-based robotic system" that was submitted and presented in EEAI2024 conference. Large Language Models (LLMs) are increasingly becoming adopted for various applications in information processing and content generation. Coupled with the diverse availability of model weights, quantization, and fine-tuning, it is desirable to find the best-performing LLM within a certain memory budget. Prompting strategy plays a similarly major role in the quality of generation, with various methods existing that attempt to induce desired behaviour, requiring a major time investment to develop. As new models with better performance-to-memory ratios become available, it may be tempting to implement them into an existing system for potential performance improvements. In this work, we explore the process of changing weights and note that weights from larger models or of different quantization precision are unable to replace the original model without modifications to the prompt contents, which in turn implies complications in developing modular, weight-agnostic systems.