LLM-Based Model Compression for Efficient on-Device Learning

Efficient deployment of large language models (LLMs) on edge devices remains a significant challenge due to their substantial computational and memory requirements. Building upon established model compression techniques for deep learning models, this work extends these methodologies to LLMs to facilitate on-device learning. We present a comprehensive approach that integrates quantization strategies to reduce model size without compromising performance. Specifically, we focus on the Microsoft Phi- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{3. 5}$</tex> model, achieving a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{5 0 \%}$</tex> reduction in its size through meticulous quantization while maintaining its original performance levels. Additionally, we developed an Android-based interface that supports the seamless import of both custom models and those available through the Hugging Face repository, thereby providing a versatile platform for deploying compressed LLMs on mobile devices. Our results demonstrate that it is feasible to implement efficient, highperforming LLMs on resource-constrained devices, paving the way for more accessible and responsive on-device intelligent applications.