RoboMed: On-Premise Medical Assistance Leveraging Large Language Models in Robotics

Large language models (LLMs) are revolutionizing numerous domains with their remarkable natural language processing (NLP) capabilities, attracting significant interest and widespread adoption. However, deploying LLMs in resource-constrained environments, such as edge computing and robotics systems without server infrastructure, while also aiming to minimize latency, presents significant challenges. Another challenge lies in delivering medical assistance to remote areas with limited healthcare facilities and infrastructure. To address this, we introduce RoboMed, an on-premise healthcare robot that utilizes compact versions of large language models (tiny-LLMs) integrated with LangChain as its backbone. Moreover, it incorporates automatic speech recognition (ASR) models for user interface, enabling efficient, edge-based preliminary medical diagnostics and support. RoboMed employs model optimizations to achieve minimal memory footprint and reduced latency during inference on embedded edge devices. The training process optimization involves low-rank adaptation (LoRA), which reduces the model's complexity without significantly impacting its performance. For fine-tuning, the LLM is trained on a diverse medical dataset compiled from online health forums, clinical case studies, and a distilled medicine corpus. This fine-tuning process utilizes reinforcement learning from human feedback (RLHF) to further enhance its domain-specific capabilities. The system is deployed on Nvidia Jetson development board and achieves 78% accuracy in medical consultations and scores 56 in USMLE benchmark, enabling an resource-efficient healthcare assistance robot that alleviates privacy concerns due to edge-based deployment, thereby empowering the community.