Develop an On - Device LLM Android Application

This paper presents the development and implementation of an Android application utilizing a Large Language Model (LLM) that operates directly on the user's device. This approach aims to address the privacy and latency concerns often associated with cloud-based LLM solutions. The application leverages a smaller, optimized LLM, GEMMA 2B, which is capable of running efficiently on mobile hardware. It incorporates a hybrid approach, utilizing on-device processing for specific queries and resorting to cloud-based LLMs for more complex tasks that require extensive knowledge bases. This paper details the system architecture, implementation details, and discusses the performance and challenges encountered during the development process.