MedBlock-Bot: A Blockchain-Enabled RAG System for Providing Feedback to Large Language Models Accessing Pediatric Clinical Guidelines

Accessing reliable clinical knowledge quickly is an everyday challenge for clinicians. Large Language Models (LLMs) can assist healthcare professionals by providing this knowledge, but their responses often deviate from expert consensus or are not up to date necessitating reliable validation and possible correction. To address this, we introduce MedBlock-Bot, an interactive Streamlit-based system integrating a blockchain-enabled Retrieval-Augmented Generation (RAG) framework for expert-driven assessment and immutable feedback storage within a permissioned consortium network. Unlike traditional feedback mechanisms that may be altered or lost, MedBlock-Bot employs smart contracts to securely store and verify any feedback, ensuring transparency and auditability. We evaluated the system using three opensource LLMs-BioMistral, HippoMistral, and LLaMa 3.1-on clinical guideline interpretation for neonates with hypoplastic left heart syndrome. Human experts assessed model responses based on accuracy and relevance, revealing variations in adherence to the guideline knowledge. Additionally, deploying the blockchain component in a local permissioned environment (Ganache) ensured efficient transaction processing and tamperproof feedback retrieval without gas cost concerns. Our results demonstrate the integration of blockchain for LLM feedback review enhancing trust, accountability, and structured knowledge retention. Clinicians can access past expert assessments for validation, while developers can leverage this feedback for potential model refinement. Taking the long-term impact into account this approach targets towards a reliable and dynamic representation of clinical knowledge and consensus. Open-Source Code: https://github.com/yaseen28/MedBlock-Bot